Caloric intake measuring system using spectroscopic and 3D imaging analysis

ABSTRACT

This invention is a caloric intake measuring system comprising: a spectroscopic sensor that collects data concerning light that is absorbed by or reflected from food, wherein this food is to be consumed by a person, and wherein this data is used to estimate the composition of this food; and an imaging device that takes images of this food from different angles, wherein these images from different angles are used to estimate the quantity of this food. Information concerning the estimated composition of the food and information concerning the estimated quantity of the food can be combined to estimate the person&#39;s caloric intake.

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application is a division of U.S. patent application Ser.No. 13/901,099 which was filed on May 23, 2013 by Robert A. Connor ofMedibotics LLC entitled “Smart Watch and Food-Imaging Member forMonitoring Food Consumption,” the entire contents thereof beingincorporated herein by reference.

FEDERALLY SPONSORED RESEARCH

Not Applicable

SEQUENCE LISTING OR PROGRAM

Not Applicable

BACKGROUND Field of Invention

This invention relates to energy balance, weight loss, and propernutrition.

Introduction to Energy Balance and Proper Nutrition

The United States population has some of the highest prevalence rates ofobese and overweight people in the world. Further, these rates haveincreased dramatically during recent decades. In the late 1990's, aroundone in five Americans was obese. Today, that figure has increased toaround one in three. It is estimated that around one in five Americanchildren is now obese. The prevalence of Americans who are generallyoverweight is estimated to be as high as two out of three.

This increase in the prevalence of Americans who are overweight or obesehas become one of the most common causes of health problems in theUnited States. Potential adverse health effects from obesity include:cancer (especially endometrial, breast, prostate, and colon cancers);cardiovascular disease (including heart attack and arterial sclerosis);diabetes (type 2); digestive diseases; gallbladder disease;hypertension; kidney failure; obstructive sleep apnea; orthopediccomplications; osteoarthritis; respiratory problems; stroke; metabolicsyndrome (including hypertension, abnormal lipid levels, and high bloodsugar); impairment of quality of life in general including stigma anddiscrimination; and even death.

There are estimated to be over a quarter-million obesity-related deathseach year in the United States. The tangible costs to American societyof obesity have been estimated at over $100 billion dollars per year.This does not include the intangible costs of human pain and suffering.Despite the considerable effort that has been focused on developing newapproaches for preventing and treating obesity, the problem is growing.There remains a serious unmet need for new ways to help people tomoderate their consumption of unhealthy food, better manage their energybalance, and lose weight in a healthy and sustainable manner.

Obesity is a complex disorder with multiple interacting causal factorsincluding genetic factors, environmental factors, and behavioralfactors. A person's behavioral factors include the person's caloricintake (the types and quantities of food which the person consumes) andcaloric expenditure (the calories that the person burns in regularactivities and exercise). Energy balance is the net difference betweencaloric intake and caloric expenditure. Other factors being equal,energy balance surplus (caloric intake greater than caloric expenditure)causes weight gain and energy balance deficit (caloric intake less thancaloric expenditure) causes weight loss.

Since many factors contribute to obesity, good approaches to weightmanagement are comprehensive in nature. Proper nutrition and managementof caloric intake are key parts of a comprehensive approach to weightmanagement. Consumption of “junk food” that is high in simple sugars andsaturated fats has increased dramatically during the past coupledecades, particularly in the United States. This has contributedsignificantly to the obesity epidemic. For many people, relying onwillpower and dieting is not sufficient to moderate their consumption ofunhealthy “junk food.” The results are dire consequences for theirhealth and well-being.

Categorization and Review of the Prior Art

Introduction to Categorization and Review of the Prior Art

One of the most important methods for addressing the growing problem ofobesity is monitoring and modifying a person's net energy balance. Aperson's net energy balance is their caloric intake minus their caloricexpenditure during a period of time. When a person's caloric intake isgreater than their caloric expenditure over time, then they have anenergy balance surplus and will gain weight. When a person's caloricintake is less than their caloric expenditure over time, then they havean energy balance deficit and will lose weight. In many nations of theworld today, including the United States, energy balance surplus andaccompanying weight gain is a serious problem for many people.

To help address this problem, there is a need for better devices andmethods for monitoring and measuring a person's caloric intake andcaloric expenditure as part of an overall approach to energy balance,weight management, and proper nutrition. There has been considerablesuccess in the development of devices and methods for automaticallymonitoring and measuring caloric expenditure. This is especially true ofrecent advances in increasingly-sophisticated wearable fitness devicesfor tracking caloric expenditure activities. These devices range fromsimple pedometers to innovative wearable fitness monitors andexercise-tracking smart watches. Most of these fitness devices include awearable accelerometer to track body motion. Some such devices also haveother wearable sensors that measure heart rate, blood pressure,temperature, electromagnetic signals from the body, and/or otherphysiological parameters.

However, measurement of the caloric intake side of the energy balanceequation is the weak link in devices and methods for energy balance andweight management. Thus far, monitoring and measuring food consumptionhas proven to be more challenging than monitoring and measuring caloricexpenditure activities. This is particularly troublesome because thereis some evidence that the obesity epidemic in the United States is beingdisproportionately caused by changes in diet.

One possible solution to address this weak link is to use an implantabledevice for monitoring and measuring a person's food consumption. In anexample, an implantable device can monitor physiological signals fromorgans along a person's gastrointestinal tract. An advantage of such animplantable device is that it does not depend on voluntary action by theperson in order to track food consumption. An implantable device canoperate automatically and continuously so that user compliance is not aproblem.

Although there is a role for implantable devices in monitoring andmeasuring food consumption, implantable devices are not for everyone.For many people, the cost and potential complications of surgery forimplantation make implantable devices less than optimal. There remains aneed for a relatively-accurate external device for monitoring andmeasuring food consumption. Accordingly, the invention disclosed hereinis an external, non-implantable device and this review focuses almostentirely on external, non-implantable devices in the prior art.

There is a central dilemma that is confounding the development ofexternal devices for monitoring and measuring a person's foodconsumption. This dilemma has not yet been solved by the prior art. Thisdilemma is the tradeoff between personal privacy and food measurementaccuracy.

On the one hand, one can create an external device that can be veryaccurate in monitoring and measuring a person's food consumption, but itwill be highly-intrusive with respect to the privacy of the person beingmonitored and other people nearby. For example, one can create a videoimaging device that a person wears continually on their head, neck, ortorso. This wearable imaging device can continually take video images ofthe space surrounding the person. Then these video images can beautomatically analyzed to detect eating events and to identify the typesof foods that the person consumes. However, continuous video monitoringof the space surrounding a person can be highly-intrusive with respectto the person's privacy and also the privacy of other people nearby.

On the other hand, one can create an external device that is relativelynon-intrusive with respect to a person's privacy, but whose accuracydepends completely on the person's compliance in using the device everytime that the person eats a meal or snack. For example, one can create amobile phone application with a menu-driven human-to-computer interfacewhich helps the person to enter information concerning food that theyconsume. Such an application can also enable a person to use the phoneto take pictures of food. These food pictures can then be analyzed toidentify the foods that the person consumes. However, the accuracy ofsuch methods based on non-wearable devices depends entirely on thedegree of consistency with which the person uses the device every timethat they eat a meal or snack. If the person neglects to use the deviceduring a midnight snack or is too embarrassed to use the device in asocial dining setting, then the accuracy of food consumption measurementsuffers.

In the invention disclosure sections which follow later in thisdisclosure, I will discuss how the invention herein solves this dilemmaof personal privacy vs. food measurement accuracy. Before disclosingthis invention, however, it is useful to first thoroughly review therelated prior art and to identify its limitations. That is what I do inthis categorization and review of the prior art. Then, when I disclosethe invention later, it will be clear how this invention addresses thelimitations of the prior art. There is a large amount of excellent andinnovative prior art in this field. Thus far, however, none of itappears to fully resolve this dilemma of personal privacy vs. foodmeasurement accuracy.

As part of this review, I have categorized the relevant prior art intogeneral categories. There are five general categories of prior art and asixth miscellaneous category. With the complexity of this field and thevolume of patents therein, seeking to categorize all relevant examplesof prior art into discrete categories is challenging. Some examples ofprior art span multiple categories and no categorization scheme isperfect.

However, even an imperfect categorization scheme can serve a usefulpurpose for reviewing the prior art. This is especially true when thereis a large quantity of relevant prior art. In the categorization andreview of the prior art herein, I have identified and classified over500 examples of prior art. Writing up individual reviews for each ofthese 500+ examples would be prohibitively lengthy and would also beless useful for the reader, who would have to wade through these 500+individual reviews. It is more efficient for the reader to be presentedwith these 500+ examples of prior art having been grouped into sixgeneral categories, wherein these six general categories are thenreviewed and discussed. To help readers who may wish to dig further intoexamples within a particular category or to second guess mycategorization scheme, I also provide relatively-detailed information oneach example of the prior art, including the patent (application) titleand date in addition to the inventors and patent (application) number.

The six categories which I use to categorize the 500+ examples of priorart for this review are as follows: (1) non-wearable devices primarilyto help measure food consumption; (2) wearable devices primarily tomonitor and measure caloric expenditure activities; (3) wearable devicesprimarily to monitor and measure food consumption; (4) wearable devicesto monitor caloric expenditure activities and to help measure foodconsumption; (5) wearable devices to monitor and measure both caloricexpenditure activities and food consumption; and (6) otherpotentially-relevant devices and methods.

In general, non-wearable devices that help a person to measure theirfood consumption depend on voluntary action by the person in associationwith each specific eating event. These non-wearable devices tend to berelatively non-intrusive with respect to privacy, but can suffer fromlow accuracy if a person does not use them consistently for every mealand snack. In general, there are few current wearable devices forautomatically detecting food consumption and these current devices arenot very accurate for identifying the specific types of foods that theperson consumes. Future generations of wearable devices will probably bemore accurate in identifying which specific foods the person consumes,but may also be highly-intrusive with respect to privacy.

The main focus of this invention is on the measurement of foodconsumption. This is currently the weak link in energy balancemeasurement. However, devices and methods for measuring caloricexpenditure activities, including pedometers and other fitness devices,are also included in this categorization scheme. This is because Ibelieve that there will be increasing convergence of food consumptionmeasurement and caloric expenditure measurement into combined energybalance devices. This makes sense because net energy balance is afunction of both energy intake and energy expenditure. This is why, forexample, I have included a category in this review for wearable fitnessdevices which monitor and measure only caloric expenditure activities,even though the primary focus of this invention is on monitoring andmeasuring food consumption. I now review each of the six categories ofprior art.

(1) Non-Wearable Devices Primarily to Help Measure Food Consumption

There are a wide variety of non-wearable devices and methods in theprior art that are intended primarily to help a person measure theirfood consumption. Since these devices are not worn by a person and donot automatically monitor the person's activities, they require sometype of voluntary action by the person in association with each eatingevent (apart from the actual act of eating).

For decades, many people manually kept track of what foods they ateand/or the associated calories (often called a “food log,” “diet log,”or “calorie counting”) using a pencil and paper. With the development ofpersonal computers, mobile electronic devices, and smart phoneapplications, much of this manual food consumption tracking has beenmade easier with menu-driven human-to-computer interfaces that helppeople to more easily enter information concerning what food they eat.Databases of common foods and their associated nutritional information(including calories) have made calorie counting easier by automaticallyassociating calories with foods entered.

Today, there are mobile phone applications that enable people tomanually enter information concerning what foods they eat. Some of theseapplications also offer automatic analysis of pictures that people takeof food in order to at least partially automate the process ofidentifying the types and amounts of food consumed. Thehuman-to-computer interfaces of such food-logging applications areevolving from keyboards and keypads to touch screens, speechrecognition, and gesture recognition. However, these approaches all relyon voluntary human action. Their accuracy is limited by: the degree towhich the person consistently uses the device for each meal or snack;and the accuracy with which the person and/or device evaluates the typesand amounts of food consumed when the device is actually used.

Although mobile phone food tracking applications are a popular form ofdevice in this category, there are a wide variety of other devices andmethods in this category beyond such mobile phone applications. Examplesof devices and methods in this category include: specialized portablecomputing devices that help a person to manual enter food consumptioninformation to create a food log; food databases that automatically linkmanually-entered foods with nutritional parameters (e.g. calories ornutrient types) associated with those foods; mobile phone applicationswith menu-driven human-to-computer interfaces for entering foodconsumption information (e.g. via keypad, touch screen, speechrecognition, or gesture recognition); imaging devices and image-analysissystems that enable automatic analysis of food pictures to identify thetypes and amounts of food in a picture; non-worn food-imaging devicesthat use bar codes or other packaging codes to identify foods; non-wornfood-imaging devices that use food logos or other packaging patterns toidentify foods; interactive food logging and meal planning websites andsoftware; smart cards and other systems based on financial transitionsthat track food purchases; devices that receive information from RFIDtags associated with food; computerized food scales, food-weighingdishes and utensils; utensils and accessories designed to track ormodify eating speed; smart food utensils or accessories that measurefood weight and/or analyze food content; food utensils and containersthat track or modify food portions; and smart food containers that tracktheir contents and/or limit access times. Specific limitations of suchdevices in the prior art include the following.

Specialized hand-held computing devices for measuring food consumptionare limited by whether a person wants to carry around a (separate)specialized electronic device, whether the person will consistently useit for every meal or snack they eat, and how skilled the person is inevaluating the amounts and types of food consumed. Food databases arelimited when a person eats foods prepared at a home or restaurant forwhich portion size and ingredients are not standardized. Mobile phoneapplications are limited by whether a person consistently uses them forevery meal or snack and by how accurate the person is in identifying theportion sizes and ingredients of non-standard foods consumed.

Non-worn imaging devices and image analysis systems are limited bywhether a person consistently uses them for every meal or snack,problems in identifying food obscured from view (such as in a cup orbowl), and foods that look similar but have different nutritionalcompositions. Also, such devices and methods can be time-consuming, easyto circumvent, and embarrassing to use in social dining situations.Further, even if a person does consistently take pictures of every mealor snack that they eat, they may be tempted to postpone foodidentification for hours or days after a meal has occurred. This cancause inaccuracy. How many chips were left in that bag in the picture?Is that a “before” or “after” picture of that half-gallon of ice cream?

Non-worn food-imaging devices that use bar codes or other packaginginformation to identify foods are limited because not all foods thatpeople eat have such codes and because people may not eat all food thatthey purchase or otherwise scan into a system. Some of the food in agiven package may be thrown out. Interactive food logging and mealplanning websites can be helpful, but they depend heavily on informationentry compliance and food consumption recall, which can be problematic.

Smart cards and other systems that are based on financial transitionsthat track food purchases are limited because people purchase food thatthey do not eat (e.g. for their family) and eat food that they do notpurchase (e.g. at home or as a guest). Also, depending on the longevityof food storage, some food may be eaten soon after purchase and some maybe eaten long afterwards. Computerized food scales and food-weighingdishes and utensils are limited because they rely on a person using themconsistently for all eating events and because some types of foodconsumption are not conducive to the use of a dish or utensil. Also,such devices and methods can be time-consuming, easy to circumvent, andembarrassing to use in social dining situations.

Utensils and accessories that are designed to track or modify eatingspeed can be useful, but depend on consistent use of the device and donot shed light on what types of food the person is eating. Smart foodutensils or accessories that measure food weight or analyze food contentare limited by the consistency of a person's use of the device. Smartfood containers that track their contents and/or limit access timesdepend on the person's exclusive use of such containers for all foodthat they eat, which can be problematic.

Specific examples of potentially-relevant prior art which appear to bemost appropriately classified into this category include the followingU.S. patents: U.S. Pat. No. 4,207,673 (DiGirolamo et al., Jun. 17, 1980,“Cuttlery”); U.S. Pat. No. 4,212,079 (Segar et al., Jul. 8, 1980,“Electronic Calorie Counter”); U.S. Pat. No. 4,218,611 (Cannon, Aug. 19,1980, “Method and Apparatus for Controlling Eating Behavior”); U.S. Pat.No. 4,321,674 (Krames et al., Mar. 23, 1982, “Nutritional ValueAccumulating and Display Device”); U.S. Pat. No. 4,650,218 (Hawke, Mar.17, 1987, “Method and Apparatus for Controlling Caloric Intake”); U.S.Pat. No. 4,686,624 (Blum et al., Aug. 11, 1987, “Portable Apparatus forAcquiring and Processing Data Relative to the Dietetics and/or theHealth of a Person”); U.S. Pat. No. 4,796,182 (Duboff, Jan. 3, 1989,“Diet Monitor and Display Device”); U.S. Pat. No. 4,875,533 (Mihara etal., Oct. 24, 1989, “Automatic Weight Detecting Device”); U.S. Pat. No.4,891,756 (Williams, Jan. 2, 1990, “Nutritional Microcomputer andMethod”); U.S. Pat. No. 4,911,256 (Attikiouzel, Mar. 27, 1990, “DieteticMeasurement Apparatus”); U.S. Pat. No. 4,914,819 (Ash, Apr. 10, 1990,“Eating Utensil for Indicating When Food May be Eaten Therewith and aMethod for Using the Utensil”); U.S. Pat. No. 4,951,197 (Mellinger, Aug.21, 1990, “Weight Loss Management System”); U.S. Pat. No. 4,975,682(Kerr et al., Dec. 4, 1990, “Meal Minder Device”); U.S. Pat. No.5,033,561 (Hettinger, Jul. 23, 1991, “Diet Control Device”); U.S. Pat.No. 5,173,588 (Harrah, Dec. 22, 1992, “Food Consumption Monitor”); U.S.Pat. No. 5,233,520 (Kretsch et al., Aug. 3, 1993, “Method and System forMeasurement of Intake of Foods, Nutrients and Other Food Components inthe Diet”); U.S. Pat. No. 5,299,356 (Maxwell, Apr. 5, 1994, “Diet EatingUtensil”); U.S. Pat. No. 5,388,043 (Hettinger, Feb. 7, 1995, “Diet andBehavioral Control Device”); U.S. Pat. No. 5,412,564 (Ecer, May 2, 1995,“System and Method for Diet Control”); U.S. Pat. No. 5,421,089 (Dubus etal., Jun. 6, 1995, “Fork with Timer”); U.S. Pat. No. 5,478,989 (Shepley,Dec. 26, 1995, “Nutritional Information System for Shoppers”); and U.S.Pat. No. 5,542,420 (Goldman et al., Aug. 6, 1996, “Personalized Methodand System for Storage, Communication, Analysis, and Processing ofHealth-Related Data”).

Additional U.S. patents which appear to be most appropriately classifiedinto this category include: U.S. Pat. No. 5,673,691 (Abrams et al., Oct.7, 1997, “Apparatus to Control Diet and Weight Using Human BehaviorModification Techniques”); U.S. Pat. No. 5,691,927 (Gump, Nov. 25, 1997,“Nutritional Aid and Method”); U.S. Pat. No. 5,704,350 (Williams, Jan.6, 1998, “Nutritional Microcomputer and Method”); U.S. Pat. No.5,729,479 (Golan, Mar. 17, 1998, “Multifunctional Diet Calculator”);U.S. Pat. No. 5,817,006 (Bergh et al., Oct. 6, 1998, “Method andApparatus for Measurement of Eating Speed”); U.S. Pat. No. 5,819,735(Mansfield et al., Oct. 13, 1998, “Device and Method for MonitoringDietary Intake of Calories and Nutrients”); U.S. Pat. No. 5,836,312(Moore, Nov. 17, 1998, “Computer-Assisted System and Method forAdjudging the Effect of Consumable Intakes on PhysiologicalParameters”); U.S. Pat. No. 5,839,901 (Karkanen, Nov. 24, 1998,“Integrated Weight Loss Control Method”); U.S. Pat. No. 5,841,115(Shepley, Nov. 24, 1998, “Nutritional Information System for Shoppers”);U.S. Pat. No. 5,890,128 (Diaz et al., Mar. 30, 1999, “Personalized HandHeld Calorie Computer (ECC)”); U.S. Pat. No. 5,989,188 (Birkhoelzer,Nov. 23, 1999, “Method and Apparatus for Determining the Energy Balanceof a Living Subject on the Basis of Energy Used and Nutrition Intake”);U.S. Pat. No. 6,024,281 (Shepley, Feb. 15, 2000, “NutritionalInformation System for Shoppers”); U.S. Pat. No. 6,032,676 (Moore, Mar.7, 2000, “Method for Correlating Consumable Intakes with PhysiologicalParameters”); U.S. Pat. No. 6,040,531 (Miller-Kovach, Mar. 21, 2000,“Process For Controlling Body Weight”); U.S. Pat. No. 6,083,006(Coffman, Jul. 4, 2000, “Personalized Nutrition Planning”); U.S. Pat.No. 6,283,914 (Mansfield et al., Sep. 4, 2001, “Device and Method forMonitoring Dietary Intake of Calories and Nutrients”); U.S. Pat. No.6,290,646 (Cosentino et al., Sep. 18, 2001, “Apparatus and Method forMonitoring and Communicating Wellness Parameters of AmbulatoryPatients”); and U.S. Pat. No. 6,336,136 (Harris, Jan. 1, 2002, “InternetWeight Reduction System”).

Further U.S. patents in this category include: U.S. Pat. No. 6,341,295(Stotler, Jan. 22, 2002, “Virtual Reality Integrated CaloricTabulator”); U.S. Pat. No. 6,454,705 (Cosentino et al., Sep. 24, 2002,“Medical Wellness Parameters Management System, Apparatus and Method”);U.S. Pat. No. 6,478,736 (Mault, Nov. 12, 2002, “Integrated CalorieManagement System”); U.S. Pat. No. 6,553,386 (Alabaster, Apr. 22, 2003,“System and Method for Computerized Visual Diet Behavior Analysis andTraining”); U.S. Pat. No. 6,694,182 (Yamazaki et al., Feb. 17, 2004,“Wearable Calorie Calculator”); U.S. Pat. No. 6,723,045 (Cosentino etal., Apr. 20, 2004, “Apparatus and Method for Monitoring andCommunicating Wellness Parameters of Ambulatory Patients”); U.S. Pat.No. 6,745,214 (Inoue et al., Jun. 1, 2004, “Calorie Control Apparatuswith Voice Recognition”); U.S. Pat. No. 6,755,783 (Cosentino et al.,Jun. 29, 2004, “Apparatus and Method for Two-Way Communication in aDevice for Monitoring and Communicating Wellness Parameters ofAmbulatory Patients”); U.S. Pat. No. 6,856,938 (Kurtz, Feb. 15, 2005,“Weight Monitoring Computer”); U.S. Pat. No. 6,878,885 (Miller-Kovach,Apr. 12, 2005, “Process for Controlling Body Weight”); U.S. Pat. No.6,917,897 (Mork, Jul. 12, 2005, “Food and Exercise Calculator”); U.S.Pat. No. 6,978,221 (Rudy, Dec. 20, 2005, “Computerized Dietetic Scale”);U.S. Pat. No. 7,044,739 (Matson, May 16, 2006, “System for ControlledNutrition Consumption”); U.S. Pat. No. 7,096,221 (Nakano, Aug. 22, 2006,“Food Information Management System”); U.S. Pat. No. 7,454,002 (Gardneret al., Nov. 18, 2008, “Integrating Personal Data CapturingFunctionality into a Portable Computing Device and a WirelessCommunication Device”); U.S. Pat. No. 7,500,937 (Hercules, Mar. 10,2009, “Diet Compliance System”); U.S. Pat. No. 7,550,683 (Daughtry, Jun.23, 2009, “Portable Digital Plate Scale”); and U.S. Pat. No. 7,577,475(Cosentino et al., Aug. 18, 2009, “System, Method, and Apparatus forCombining Information from an Implanted Device with Information from aPatient Monitoring Apparatus”).

Further U.S. patents in this category include: U.S. Pat. No. 7,736,318(Cosentino et al., Jun. 15, 2010, “Apparatus and Method for Monitoringand Communicating Wellness Parameters of Ambulatory Patients”); U.S.Pat. No. 7,769,635 (Simons-Nikolova, Aug. 3, 2010, “Weight ManagementSystem with Simple Data Input”); U.S. Pat. No. 7,857,730 (Dugan, Dec.28, 2010, “Methods and Apparatus for Monitoring and Encouraging Healthand Fitness”); U.S. Pat. No. 7,899,709 (Allard et al., Mar. 1, 2011,“System and Method for Identification and Tracking of Food Items”); U.S.Pat. No. 7,949,506 (Hill et al., May 24, 2011, “Method for Determiningand Compensating for a Weight Loss Energy Gap”); U.S. Pat. No. 7,956,997(Wang et al., Jun. 7, 2011, “Systems and Methods for Food SafetyDetection”); U.S. Pat. No. 7,999,674 (Kamen, Aug. 16, 2011, “Device andMethod for Food Management”); U.S. Pat. No. 8,075,451 (Dugan, Dec. 13,2011, “Methods and Apparatus for Monitoring and Encouraging Health andFitness”); U.S. Pat. No. 8,087,937 (Peplinski et al., Jan. 3, 2012,“System and Method for Monitoring Weight and Nutrition”); U.S. Pat. No.8,229,676 (Hyde et al., Jul. 24, 2012, “Food Content Detector”); U.S.Pat. No. 8,285,488 (Hyde et al., Oct. 9, 2012, “Food Content Detector”);U.S. Pat. No. 8,290,712 (Hyde et al., Oct. 16, 2012, “Food ContentDetector”); U.S. Pat. No. 8,294,581 (Kamen, Oct. 23, 2012, “Device andMethod for Food Management”); U.S. Pat. No. 8,299,930 (Schmid-Schonbeinet al., Oct. 30, 2012, “Devices, Systems and Methods to Control CaloricIntake”); U.S. Pat. No. 8,321,141 (Hyde et al., Nov. 27, 2012, “FoodContent Detector”); U.S. Pat. No. 8,330,057 (Sharawi et al., Dec. 11,2012, “System and Method for Weighing Food and Calculating CalorieContent Thereof”); U.S. Pat. No. 8,337,367 (Dugan, Dec. 25, 2013,“Methods and Apparatus for Monitoring and Encouraging Health andFitness”); U.S. Pat. No. 8,345,930 (Tamrakar et al., Jan. 1, 2013,“Method for Computing Food Volume in a Method for Analyzing Food”); U.S.Pat. No. 8,355,875 (Hyde et al., Jan. 15, 2013, “Food ContentDetector”); U.S. Pat. No. 8,363,913 (Boushey et al., Jan. 29, 2013,“Dietary Assessment System and Method”); U.S. Pat. No. 8,386,185 (Hydeet al., May 20, 2010, “Food Content Detector”); U.S. Pat. No. 8,392,123(Hyde et al., May 20, 2010, “Food Content Detector”); U.S. Pat. No.8,392,124 (Hyde et al., May 20, 2010, “Food Content Detector”); U.S.Pat. No. 8,392,125 (Hyde et al., Mar. 5, 2013, “Food Content Detector”);U.S. Pat. No. 8,396,672 (Hyde et al., Mar. 12, 2013, “Food ContentDetector”); and U.S. Pat. No. 8,438,038 (Cosentino et al., May 7, 2013,“Weight Loss or Weight Management System”).

Specific examples of potentially-relevant prior art which appear to bemost appropriately classified into this category also include thefollowing U.S. patent applications: 20020062069 (Mault, May 23, 2002,“System and Method of Integrated Calorie Management Using InteractiveTelevision”); 20020124017 (Mault, Sep. 5, 2002, “Personal DigitalAssistant with Food Scale Accessory”); 20020167863 (Davis et al., Nov.14, 2002, “Portable, Compact Device to Monitor Rate and Quantity ofDietary Intake to Control Body Weight”); 20030076983 (Cox, Apr. 24,2003, “Personal Food Analyzer”); 20030152607 (Mault, Aug. 14, 2003,“Caloric Management System and Method with Voice Recognition”);20030163354 (Shamoun, Aug. 28, 2003, “Device for Collecting andAnalyzing Nutritional Data and Method Therefor”); 20030165799 (Bisogno,Sep. 4, 2003, “Computer Program, Method, and System for MonitoringNutrition Content of Consumables and for Facilitating Menu Planning”);20030219513 (Gordon, Nov. 27, 2003, “Personal Nutrition ControlMethod”); 20050008994 (Bisogno, Jan. 13, 2005, “Computer Program,Method, and System for Monitoring Nutrition Content of Consumables andfor Facilitating Menu Planning”); 20050011367 (Crow, Jan. 20, 2005,“Portion Control Serving Utensils”); 20050014111 (Matson, Jan. 20, 2005,“System for Controlled Nutrition Consumption”); 20050153052 (Williams etal., Jul. 14, 2005, “Food and Beverage Quality Sensor”); 20050184148(Perlman, Aug. 25, 2005, “Scale Having Nutritional InformationReadouts”); 20050247213 (Slilaty, Nov. 10, 2005, “Method of IdentifyingParticular Attributes of Food Products Consistent with Consumer Needsand/or Desires”); and 20050266385 (Bisogno, Dec. 1, 2005, “ComputerProgram, Method, and System for Monitoring Nutrition Content ofConsumables and for Facilitating Menu Planning”).

Additional U.S. patent applications which appear to be mostappropriately classified into this category include: 20060036395 (Shayaet al., Feb. 16, 2006, “Method and Apparatus for Measuring andControlling Food Intake of an Individual”); 20060074716 (Tilles et al.,Apr. 6, 2006, “System and Method for Providing Customized Interactiveand Flexible Nutritional Counseling”); 20060189853 (Brown, Aug. 24,2006, “Method and System for Improving Adherence with a Diet Program orOther Medical Regimen”); 20060229504 (Johnson, Oct. 12, 2006, “Methodsand Systems for Lifestyle Management”); 20060263750 (Gordon, Nov. 23,2006, “Personal Nutrition Control Devices”); 20070021979 (Cosentino etal., Jan. 25, 2007, “Multiuser Wellness Parameter Monitoring System”);20070027366 (Osburn, Feb. 1, 2007, “Device and System for Entering andMonitoring Dietary Data”); 20070028453 (Crow, Feb. 8, 2007, “PortionControl Serving Utensils”); 20070030339 (Findlay et al., Feb. 8, 2007,“Method, System and Software for Monitoring Compliance”); 20070050058(Zuziak et al., Mar. 1, 2007, “Placemat for Calculating and MonitoringCalorie Intake”); 20070059672 (Shaw, Mar. 15, 2007, “Nutrition TrackingSystems and Methods”); 20070089335 (Smith et al., Apr. 26, 2007,“Nutrient Consumption/Expenditure Planning and Tracking Apparatus Systemand Method”); 20070098856 (LePine, May 3, 2007, “Mealtime EatingRegulation Device”); 20070173703 (Lee et al., Jul. 26, 2007, “Method,Apparatus, and Medium for Managing Weight by Using Calorie ConsumptionInformation”); 20070179355 (Rosen, Aug. 2, 2007, “Mobile Self-ManagementCompliance and Notification Method, System and Computer ProgramProduct”); 20070208593 (Hercules, Sep. 6, 2007, “Diet ComplianceSystem”); 20080019122 (Kramer, Jan. 24, 2008, “Foodware System HavingSensory Stimulating, Sensing and/or Data Processing Components”);20080060853 (Davidson et al., Mar. 13, 2008, “Scales DisplayingNutritional Information”); and 20080255955 (Simons-Nikolova, Oct. 16,2008, “Weight Management System with Simple Data Input”).

Further U.S. patent applications in this category include: 20080267444(Simons-Nikolova, Oct. 30, 2008, “Modifying a Person's Eating andActivity Habits”); 20080270324 (Allard et al., Oct. 30, 2008, “Systemand Method for Identification and Tracking of Food Items”); 20080276461(Gold, Nov. 13, 2008, “Eating Utensil Capable of Automatic BiteCounting”); 20090112800 (Athsani, Apr. 30, 2009, “System and Method forVisual Contextual Search”); 20090176526 (Altman, Jul. 9, 2009,“Longitudinal Personal Health Management System Using Mobile DataCapture”); 20090191514 (Barnow, Jul. 30, 2009, “Calorie Counter”);20090219159 (Morgenstern, Sep. 3, 2009, “Method and System for anElectronic Personal Trainer”); 20090253105 (Lepine, Oct. 8, 2009,“Device for Regulating Eating by Measuring Potential”); 20100003647(Brown et al., Jan. 7, 2010, “System and Method for Automated MealRecommendations”); 20100057564 (Godsey et al., Mar. 4, 2010, “System andMethod for Fitness Motivation”); 20100062119 (Miller-Kovach, Mar. 11,2010, “Processes and Systems for Achieving and Assisting in ImprovedNutrition”); 20100062402 (Miller-Kovach, Mar. 11, 2010, “Processes andSystems Using and Producing Food Healthfulness Data Based on LinearCombinations of Nutrients”); 20100080875 (Miller-Kovach, Apr. 1, 2010,“Processes and Systems for Achieving and Assisting in Improved NutritionBased on Food Energy Data and Relative Healthfulness Data”); 20100109876(Schmid-Schonbein et al., May 6, 2010, “Devices, Systems and Methods toControl Caloric Intake”); 20100111383 (Boushey et al., May 6, 2010,“Dietary Assessment System and Method”); 20100125176 (Hyde et al., May20, 2010, “Food Content Detector”); and 20100125177 (Hyde et al., May20, 2010, “Food Content Detector”).

Further U.S. patent applications in this category include: 20100125178(Hyde et al., May 20, 2010, “Food Content Detector”); 20100125179 (Hydeet al., May 20, 2010, “Food Content Detector”); 20100125180 (Hyde etal., May 20, 2010, “Food Content Detector”); 20100125181 (Hyde et al.,May 20, 2010, “Food Content Detector”); 20100125417 (Hyde et al., May20, 2010, “Food Content Detector”); 20100125418 (Hyde et al., May 20,2010, “Food Content Detector”); 20100125419 (Hyde et al., May 20, 2010,“Food Content Detector”); 20100125420 (Hyde et al., May 20, 2010, “FoodContent Detector”); 20100173269 (Puri et al., Jul. 8, 2010, “FoodRecognition Using Visual Analysis and Speech Recognition”); 20100191155(Kim et al., Jul. 29, 2010, “Apparatus for Calculating Calories Balanceby Classifying User's Activity”); 20100205209 (Jokinen, Aug. 12, 2010,“Method and System for Monitoring a Personal Intake”); 20100332571(Healey et al., Dec. 30, 2010, “Device Augmented Food Identification”);20110124978 (Williams, May 26, 2011, “Health and Fitness System”);20110182477 (Tamrakar et al., Jul. 28, 2011, “Method for Computing FoodVolume in a Method for Analyzing Food”); 20110184247 (Contant et al.,Jul. 28, 2011, “Comprehensive Management of Human Health”); 20110281245(Mansour, Nov. 17, 2011, “System for Regulating Caloric Intake andMethod for Using Same”); 20110318717 (Adamowicz, Dec. 29, 2011,“Personalized Food Identification and Nutrition Guidance System”);20120031805 (Stolarczyk, Feb. 9, 2012, “Daily Meal Planning System”);20120055718 (Chen, Mar. 8, 2012, “Electronic Scale for Recording HealthAdministration Data”); and 20120072233 (Hanlon et al., Mar. 22, 2012,“Medical Health Information System for Health Assessment, WeightManagement and Meal Planning”).

Further U.S. patent applications in this category include: 20120077154(Highet et al., Mar. 29, 2012, “Incrementally-Sized Standard-SizedEating-Ware System for Weight Management”); 20120083669 (Abujbara, Apr.5, 2012, “Personal Nutrition and Wellness Advisor”); 20120096405 (Seo,Apr. 19, 2012, “Apparatus and Method for Diet Management”); 20120115111(Lepine, May 10, 2012, “Mealtime Eating Regulation Device”); 20120126983(Breibart, May 24, 2012, “Method and Associated Device for PersonalWeight Control or Weight Loss”); 20120144912 (Kates et al., Jun. 14,2012, “Portion Control System for Weight Loss and Maintenance”);20120170801 (De Oliveira et al., Jul. 5, 2012, “System for FoodRecognition Method Using Portable Devices Having Digital Cameras”);20120178065 (Naghavi et al., Jul. 12, 2012, “Advanced Button Applicationfor Individual Self-Activating and Monitored Control System in WeightLoss Program”); 20120179665 (Baarman et al., Jul. 12, 2012, “HealthMonitoring System”); 20120221495 (Landers, Aug. 30, 2012, “DigitalWeight Loss Aid”); 20120295233 (Cooperman, Nov. 22, 2012, “ComputerizedSystem and Method for Monitoring Food Consumption”); 20120315609(Miller-Kovach et al., Dec. 13, 2012, “Methods and Systems for WeightControl by Utilizing Visual Tracking of Living Factor(s)”); 20120321759(Marinkovich et al., Dec. 20, 2012, “Characterization of Food Materialsby Optomagnetic Fingerprinting”); 20130006063 (Wang, Jan. 3, 2013,“Physiological Condition, Diet and Exercise Plan Recommendation andManagement System”); 20130006802 (Dillahunt et al., Jan. 3, 2013,“Generating a Location-Aware Preference and Restriction-Based CustomizedMenu”); and 20130006807 (Bai et al., Jan. 3, 2013, “Guideline-Based FoodPurchase Management”).

Further U.S. patent applications in this category include: 20130012788(Horseman, Jan. 10, 2013, “Systems, Computer Medium andComputer-Implemented Methods for Monitoring and Improving BiometricHealth of Employees”); 20130012789 (Horseman, Jan. 10, 2013, “Systems,Computer Medium and Computer-Implemented Methods for Monitoring andImproving Biomechanical Health of Employees”); 20130012790 (Horseman,Jan. 10, 2013, “Systems, Computer Medium and Computer-ImplementedMethods for Monitoring and Improving Health and Productivity ofEmployees”); 20130012802 (Horseman, Jan. 10, 2013, “Systems, ComputerMedium and Computer-Implemented Methods for Monitoring and ImprovingCognitive and Emotive Health of Employees”); 20130013331 (Horseman, Jan.10, 2013, “Systems, Computer Medium and Computer-Implemented Methods forMonitoring Health of Employees Using Mobile Devices”); 20130043997(Cosentino et al., Feb. 21, 2013, “Weight Loss Or Weight ManagementSystem”); 20130043997 (Cosentino et al., Feb. 21, 2013, “Weight Loss orWeight Management System”); 20130045467 (Kamen, Feb. 21, 2013, “Deviceand Method for Food Management”); 20130090565 (Quy, Apr. 11, 2013,“Method and Apparatus for Monitoring Exercise with Wireless InternetConnectivity”); 20130091454 (Papa et al., Apr. 11, 2013, “PhysicalHealth Application and Method for Implementation”); 20130105565(Kamprath, May 2, 2013, “Nutritional Information System”); 20130108993(Katz; David L. May 2, 2013, “Method and System for Scoring a Diet”);and 20130113933 (Boushey et al., May 9, 2013, “Dietary Assessment Systemand Method”). Prior art which appears to be most appropriatelyclassified into this category also includes WO 1997028738 (Zuabe, Aug.14, 1997, “Portable Apparatus for Monitoring Food Intake”).

(2) Wearable Devices Primarily to Monitor and Measure CaloricExpenditure Activities

Although the main focus of this invention is on the monitoring andmeasurement of food consumption, there are reasons why I have alsoincluded this category for wearable fitness devices which primarily orexclusively monitor and measure caloric expenditure activities. First,there has been more progress in the prior art toward automaticmonitoring and measuring of caloric expenditure activities than therehas been toward automatic monitoring and measuring of caloric intake.There can be lessons learned and cross-over technology between the twosides of the energy balance equation. Second, there will probably beincreasing convergence of caloric expenditure and intake measurementinto combined energy balance devices. For example, especially forwearable fitness devices that include an accelerometer, it may bepossible to use this accelerometer to also monitor for possible eatingevents (especially if the device is worn on a body member that moveswhen a person is eating).

Most devices and methods in this category include a wearableaccelerometer which is used to analyze a person's movements and/orestimate their caloric expenditure. Some of the more-sophisticateddevices also include wearable sensors that measure heart rate, bloodpressure, temperature, electromagnetic signals from the body, and/orother physiologic parameters. Some fitness monitors also supplement anaccelerometer with an altimeter and GPS functionality.

Most devices and methods in this category measure motion from onelocation on a person's body, unlike the full-body “motion capture”technology that is used for animation in motion pictures. There ismovement toward the use of full-body motion recognition in gamingsystems for measuring caloric expenditure, but this is not currentlywearable and portable technology. Most wearable and portable technologyis still based on measurement of body movement from one location on aperson's body. Accordingly, some types of calorie burning activities aremore accurately measured than others. For example, although fidgetingburns calories, an accelerometer attached to a person's torso, on theirneck, or in their pocket will not measure this type of calorie-burningactivity very well.

Although devices and methods in this category can be an important partof monitoring and measuring energy balance, they currently provide verylittle (if any) monitoring or measurement of a person's foodconsumption. I did my best to thoroughly review the 500+ examples ofprior art and to place into this category those examples which appear tofocus primarily on measuring caloric expenditure activities with little(or no) mention of food, nutrition, eating, or caloric intake. However,if I missed a reference to measuring food consumption or caloric intakein the details of one of these examples, then that example could bebetter classified into either category 4 or 5 which follow. Bydefinition, prior art in this category is very limited in terms ofmonitoring or measuring food consumption.

Specific examples of potentially-relevant prior art which appear to bemost appropriately classified into this category include the followingU.S. patents: U.S. Pat. No. 4,757,453 (Nasiff, Jul. 12, 1988, “BodyActivity Monitor Using Piezoelectric Transducers on Arms and Legs”);U.S. Pat. No. 5,038,792 (Mault, Aug. 13, 1991, “Oxygen ConsumptionMeter”); U.S. Pat. No. 6,135,951 (Richardson et al., Oct. 24, 2000,“Portable Aerobic Fitness Monitor for Walking and Running”); U.S. Pat.No. 6,498,994 (Vock et al., Dec. 24, 2002, “Systems and Methods forDetermining Energy Experienced by a User and Associated With Activity”);U.S. Pat. No. 6,527,711 (Stivoric et al., Mar. 4, 2003, “Wearable HumanPhysiological Data Sensors and Reporting System Therefor”); U.S. Pat.No. 6,856,934 (Vock et al., Feb. 15, 2005, “Sport Monitoring Systems andAssociated Methods”); U.S. Pat. No. 7,054,784 (Flentov et al., May 30,2006, “Sport Monitoring Systems”); U.S. Pat. No. 7,057,551 (Vogt, Jun.6, 2006, “Electronic Exercise Monitor and Method Using a LocationDetermining Component and a Pedometer”); U.S. Pat. No. 7,153,262(Stivoric et al., Dec. 26, 2006, “Wearable Human Physiological DataSensors and Reporting System Therefor”); U.S. Pat. No. 7,255,437 (Howellet al., Aug. 14, 2007, “Eyeglasses with Activity Monitoring”); U.S. Pat.No. 7,373,820 (James, May 20, 2008, “Accelerometer for Data Collectionand Communication”); U.S. Pat. No. 7,398,151 (Burrell et al., Jul. 8,2008, “Wearable Electronic Device”); U.S. Pat. No. 7,401,918 (Howell etal., Jul. 22, 2008, “Eyeglasses with Activity Monitoring”); U.S. Pat.No. 7,438,410 (Howell et al., Oct. 21, 2008, “Tethered ElectricalComponents for Eyeglasses”); U.S. Pat. No. 7,451,056 (Flentov et al.,Nov. 11, 2008, “Activity Monitoring Systems and Methods”); U.S. Pat. No.7,481,531 (Howell et al., Jan. 27, 2009, “Eyeglasses with UserMonitoring”); U.S. Pat. No. 7,512,515 (Vock et al., Mar. 31, 2009,“Activity Monitoring Systems and Methods”); U.S. Pat. No. 7,640,804(Daumer et al., Jan. 5, 2010, “Apparatus for Measuring Activity”); andU.S. Pat. No. 7,717,866 (Damen, May 18, 2010, “Portable DeviceComprising an Acceleration Sensor and Method of Generating Instructionsor Advice”).

Additional U.S. patents which appear to be most appropriately classifiedinto this category include: U.S. Pat. No. 7,805,196 (Miesel et al., Sep.28, 2010, “Collecting Activity Information to Evaluate Therapy”); U.S.Pat. No. 7,841,966 (Aaron et al., Nov. 30, 2010, “Methods, Systems, andProducts for Monitoring Athletic Performance”); U.S. Pat. No. 7,980,997(Thukral et al., Jul. 19, 2011, “System for Encouraging a User toPerform Substantial Physical Activity”); U.S. Pat. No. 8,021,297 (Aerts,Sep. 20, 2011, “Wearable Device”); U.S. Pat. No. 8,033,959 (Oleson etal., Oct. 11, 2011, “Portable Fitness Monitoring Systems, andApplications Thereof”); U.S. Pat. No. 8,068,858 (Werner et al., Nov. 29,2011, “Methods and Computer Program Products for Providing Informationabout a User During a Physical Activity”); U.S. Pat. No. 8,162,804(Tagliabue, Apr. 24, 2012, “Collection and Display of AthleticInformation”); U.S. Pat. No. 8,184,070 (Taubman, May 22, 2012, “Methodand System for Selecting a User Interface for a Wearable ComputingDevice”); U.S. Pat. No. 8,244,278 (Werner et al., Aug. 14, 2012,“Portable Fitness Systems, and Applications Thereof”); U.S. Pat. No.8,265,907 (Nanikashvili et al., Sep. 11, 2012, “System and a Method forPhysiological Monitoring”); U.S. Pat. No. 8,352,211 (Vock et al., Jan.8, 2013, “Activity Monitoring Systems and Methods”); U.S. Pat. No.8,370,549 (Burton et al., Feb. 5, 2013, “Wearable Device Assembly HavingAthletic Functionality”); U.S. Pat. No. 8,378,811 (Crump et al., Feb.19, 2013, “Mobile Wireless Customizable Health and Condition Monitor”);U.S. Pat. No. 8,403,845 (Stivoric et al., Mar. 26, 2013, “Wearable HumanPhysiological and Environmental Data Sensors and Reporting SystemTherefor”); U.S. Pat. No. 8,408,436 (Berry et al., Apr. 2, 2013,“Wearable Device Assembly Having Athletic Functionality”); U.S. Pat. No.8,416,102 (Yin Apr. 9, 2013, “Activity Monitoring System Insensitive toAccelerations Induced by External Motion Factors”); and U.S. Pat. No.8,421,620 (Boyd et al., Apr. 16, 2013, “Electronically TriggeredPersonal Athletic Device”).

Specific examples of potentially-relevant prior art which appear to bemost appropriately classified into this category also include thefollowing U.S. patent applications: 20110288379 (Wu, Nov. 24, 2011,“Body Sign Dynamically Monitoring System”); 20120004883 (Vock et al.,Jan. 5, 2012, “Activity Monitoring Systems and Methods”); 20120150074(Yanev et al., Jun. 14, 2012, “Physical Activity Monitoring System”);20120150327 (Altman et al., Jun. 14, 2012, “System, Method, Apparatus,or Computer Program Product for Exercise and Personal Security”);20120245716 (Srinivasan et al., Sep. 27, 2012, “Activity MonitoringDevice and Method”); 20120251079 (Meschter et al., Oct. 4, 2012,“Systems and Methods for Time-Based Athletic Activity Measurement andDisplay”); 20120253485 (Weast et al., Oct. 4, 2012, “Wearable DeviceHaving Athletic Functionality”); 20120258433 (Hope et al., Oct. 11,2012, “Fitness Monitoring Methods, Systems, and Program Products, andApplications Thereof”); 20120268592 (Aragones et al., Oct. 25, 2012,“Processing Data of a User Performing an Athletic Activity to EstimateEnergy Expenditure”); 20120274508 (Brown et al., Nov. 1, 2012, “AthleticWatch”); 20120274554 (Kinoshita et al., Nov. 1, 2012, “Body MovementDetection Device and Display Control Method of Body Movement DetectionDevice”); 20120283855 (Hoffman et al., Nov. 8, 2012, “Monitoring FitnessUsing a Mobile Device”); 20120289867 (Kasama, Nov. 15, 2012, “StateDetermining Device and State Determination Method”); 20120290109(Engelberg et al., Nov. 15, 2012, “Methods and Systems for EncouragingAthletic Activity”); 20120310971 (Tran, Dec. 6, 2012, “Fitness Device”);20120316406 (Rahman et al., Dec. 13, 2012, “Wearable Device and Platformfor Sensory Input”); 20120316455 (Rahman et al., Dec. 13, 2012,“Wearable Device and Platform for Sensory Input”); and 20120316456(Rahman et al., Dec. 13, 2012, “Sensory User Interface”).

Additional U.S. patent applications which appear to be mostappropriately classified into this category include: 20120316471 (Rahmanet al., Dec. 13, 2012, “Power Management in a Data-Capable Strapband”);20120316661 (Rahman et al., Dec. 13, 2012, “Media Device, Application,and Content Management Using Sensory Input”); 20120317430 (Rahman etal., Dec. 13, 2012, “Power Management in a Data-Capable Strapband”);20120323346 (Ashby et al., Dec. 20, 2012, “Portable Physical ActivitySensing System”); 20120323496 (Burroughs et al., Dec. 20, 2012,“Tracking of User Performance Metrics During a Workout Session”);20130005534 (Rosenbaum, Jan. 3, 2013, “Instrumented Article of Fitnessand Method of Determining Caloric Requirements”); 20130006583 (Weast etal., Jan. 3, 2013, “Sensor-Based Athletic Activity Measurements”);20130041617 (Pease et al., Feb. 14, 2013, “Systems and Methods forMonitoring Athletic Performance”); 20130052623 (Thukral et al., Feb. 28,2013, “System for Encouraging a User to Perform Substantial PhysicalActivity”); 20130053990 (Ackland, Feb. 28, 2013, “Classification Systemand Method”); 20130073368 (Squires, Mar. 21, 2013, “IncentivizingPhysical Activity”); 20130083009 (Geisner et al., Apr. 4, 2013,“Exercising Applications for Personal Audio/Visual System”); 20130102387(Barsoum et al., Apr. 25, 2013, “Calculating Metabolic Equivalence witha Computing Device”); 20130103416 (Amigo et al., Apr. 25, 2013, “Systemsand Methods for Activity Evaluation”); 20130106603 (Weast et al., May 2,2013, “Wearable Device Assembly Having Athletic Functionality”);20130106684 (Weast et al., May 2, 2013, “Wearable Device Assembly HavingAthletic Functionality”); 20130110011 (McGregor et al., May 2, 2013,“Method of Monitoring Human Body Movement”); 20130110264 (Weast et al.,May 2, 2013, “Wearable Device Having Athletic Functionality”);20130115583 (Gordon et al., May 9, 2013, “User Interface for RemoteJoint Workout Session”); and 20130115584 (Gordon et al., May 9, 2013,“User Interface and Fitness Meters for Remote Joint Workout Session”).

(3) Wearable Devices Primarily to Monitor and Measure Food Consumption

Devices and methods in the previous category (category 2) focusprimarily or exclusively on the caloric expenditure side of the energybalance equation. Devices and methods in this present category (category3) focus primarily or exclusively on the caloric intake side of energybalance. Prior art in this present category includes wearable devicesthat are primarily for monitoring and measuring food consumption. Ingeneral, there has been less progress on the caloric intake side of theequation. Also, most devices that offer automatic monitoring andmeasurement of food consumption also offer at least some monitoring andmeasurement of caloric expenditure activities. Wearable devices thatoffer at least some measurement of both food consumption and caloricexpenditure activities are classified in categories 4 or 5 which follow.

Examples of devices and methods in this category include: wearableaccelerometers or other motion sensors that detect body motionsassociated with eating (e.g. particular patterns of hand movements ormouth movements); wearable heart rate, blood pressure, and/orelectromagnetic body signal monitors that are used to detect eatingevents; wearable thermal energy sensors that are used to detect eatingevents; wearable glucose monitors that are used to detect eating eventsand provide some information about the nutritional composition of foodconsumed; wearable body fluid sampling devices such as continuousmicro-sampling blood analysis devices; wearable sound sensors thatdetect body sounds or environmental sounds associated with eating events(e.g. chewing sounds, swallowing sounds, gastrointestinal organ sounds,and verbal food orders); and wearable cameras that continually takevideo images of the space surrounding the person wherein these videoimages are analyzed to detect eating events and identify foods consumed.

As mentioned previously, the prior art for devices and methods forwearable food consumption monitoring is generally less well-developedthan the prior art for wearable caloric expenditure monitoring. Most ofthe prior art in this category offers some indication of eating events,but not very good identification of the specific amounts and types offood that a person eats. For example, a wrist-mounted accelerometer maybe able to generally count the number of mouthfuls of food that a personconsumes, but does not shed light on what type of food that person iseating. The same limitation is generally true for wearable heart rate,blood pressure, temperature, and electromagnetic monitors. Wearablecontinuous glucose monitors can provide more information than thepreceding monitors, but still fall far short of creating a complete foodconsumption log for energy balance and nutritional purposes.

Wearable video imaging devices that continually record video images ofthe space surrounding a person have the potential to offer much moreaccurate detection of eating and identification of the types and amountsof food consumed. However, as we have discussed, such devices can alsobe highly-intrusive with respect to the privacy of the person beingmonitored and also everyone around them. This privacy concern can be aserious limitation for the use of a wearable video imaging device formonitoring and measuring food consumption. Since most developers ofwearable video imaging devices appear to be developing such devices formany more applications than just monitoring food consumption, most suchprior art is not categorized into this category.

Specific examples of potentially-relevant prior art which appear to bemost appropriately classified into this category include the followingU.S. patents: U.S. Pat. No. 4,100,401 (Tutt et al., Jul. 11, 1978,“Calorie Calculator-Chronometer”); U.S. Pat. No. 4,192,000 (Lipsey, Mar.4, 1980, “Electronic Calorie Counter”); U.S. Pat. No. 4,509,531 (Ward,Apr. 9, 1985, “Personal Physiological Monitor”); U.S. Pat. No. 4,823,808(Clegg et al., Apr. 25, 1989, “Method for Control of Obesity Overweightand Eating Disorders”); U.S. Pat. No. 4,965,553 (DelBiondo et al., Oct.23, 1990, “Hand-Near-Mouth Warning Device”); U.S. Pat. No. 5,050,612(Matsumura, Sep. 24, 1991, “Device for Computer-Assisted Monitoring ofthe Body”); U.S. Pat. No. 5,067,488 (Fukada et al., Nov. 26, 1991,“Mastication Detector and Measurement Apparatus and Method of MeasuringMastication”); U.S. Pat. No. 5,263,491 (Thornton, Nov. 23, 1993,“Ambulatory Metabolic Monitor”); U.S. Pat. No. 5,398,688 (Laniado, Mar.21, 1995, “Method, System and Instrument for Monitoring Food Intake”);U.S. Pat. No. 5,424,719 (Ravid, Jun. 13, 1995, “Consumption Control”);U.S. Pat. No. 5,497,772 (Schulman et al., Mar. 12, 1996, “GlucoseMonitoring System”); U.S. Pat. No. 5,563,850 (Hanapole, Oct. 8, 1996,“Food Intake Timer”); U.S. Pat. No. 6,135,950 (Adams, Oct. 24, 2000,“E-fit Monitor”); U.S. Pat. No. 6,249,697 (Asano, Jun. 19, 2001,“Electrogastrograph and Method for Analyzing Data Obtained by theElectrogastrograph”); U.S. Pat. No. 6,425,862 (Brown, Jul. 30, 2002,“Interactive Furniture for Dieters”); U.S. Pat. No. 6,508,762 (Karnieli,Jan. 21, 2003, “Method for Monitoring Food Intake”); and U.S. Pat. No.6,893,406 (Takeuchi et al., May 17, 2005, “Mastication MonitoringDevice”).

Additional U.S. patents which appear to be most appropriately classifiedinto this category include: U.S. Pat. No. 7,855,936 (Czarnek et al.,Dec. 21, 2010, “Diet Watch”); U.S. Pat. No. 7,878,975 (Liljeryd et al.,Feb. 1, 2011, “Metabolic Monitoring, a Method and Apparatus forIndicating a Health-Related Condition of a Subject”); U.S. Pat. No.8,112,281 (Yeung et al., Feb. 7, 2012, “Accelerometer-Based Control ofWearable Audio Recorders”); U.S. Pat. No. 8,158,082 (Imran, Apr. 17,2012, “Micro-Fluidic Device”); U.S. Pat. No. 8,236,242 (Drucker et al.,Aug. 7, 2012, “Blood Glucose Tracking Apparatus and Methods”); U.S. Pat.No. 8,275,438 (Simpson et al., Sep. 25, 2012, “Analyte Sensor”); U.S.Pat. No. 8,280,476 (Jina, Oct. 2, 2012, “Devices, Systems, Methods andTools for Continuous Glucose Monitoring”); U.S. Pat. No. 8,287,453 (Liet al., Oct. 16, 2012, “Analyte Sensor”); U.S. Pat. No. 8,298,142(Simpson et al., Oct. 30, 2012, “Analyte Sensor”); U.S. Pat. No.8,310,368 (Hoover et al., Nov. 13, 2012, “Weight Control Device”); U.S.Pat. No. 8,369,919 (Kamath et al., Feb. 5, 2013, “Systems and Methodsfor Processing Sensor Data”); U.S. Pat. No. 8,417,312 (Kamath et al.,Apr. 9, 2013, “Systems and Methods for Processing Sensor Data”); andU.S. Pat. No. 8,423,113 (Shariati et al., Apr. 16, 2013, “Systems andMethods for Processing Sensor Data”); U.S. Pat. No. 8,438,163 (Li etal., May 7, 2013, “Automatic Learning of Logos for Visual Recognition”).

Specific examples of potentially-relevant prior art which appear to bemost appropriately classified into this category also include thefollowing U.S. patent applications: 20020022774 (Karnieli, Feb. 21,2002, “Method for Monitoring Food Intake”); 20040073142 (Takeuchi etal., Apr. 15, 2004, “Mastication Monitoring Device”); 20050283096 (Chauet al., Dec. 22, 2005, “Apparatus and Method for Detecting SwallowingActivity”); 20060197670 (Breibart, Sep. 7, 2006, “Method and AssociatedDevice for Personal Weight Control”); 20080137486 (Czarenk et al., Jun.12, 2008, “Diet Watch”); 20080262557 (Brown, Oct. 23, 2008, “ObesityManagement System”); 20100194573 (Hoover et al., Aug. 5, 2010, “WeightControl Device”); 20100240962 (Contant, Sep. 23, 2010, “Eating Utensilto Monitor and Regulate Dietary Intake”); 20120078071 (Bohm et al., Mar.29, 2012, “Advanced Continuous Analyte Monitoring System”); 20120149996(Stivoric et al., Jun. 14, 2012, “Method and Apparatus for ProvidingDerived Glucose Information Utilizing Physiological and/or ContextualParameters”); 20120149996 (Stivoric et al., Jun. 14, 2012, “Method andApparatus for Providing Derived Glucose Information UtilizingPhysiological and/or Contextual Parameters”); 20120191052 (Rao, Jul. 26,2012, “Intelligent Activated Skin Patch System”); 20120194418 (Osterhoutet al., Aug. 2, 2012, “AR Glasses with User Action Control and EventInput Based Control of Eyepiece Application”); 20120194419 (Osterhout etal., Aug. 2, 2012, “AR Glasses with Event and User Action Control ofExternal Applications”); and 20120194420 (Osterhout et al., Aug. 2,2012, “AR Glasses with Event Triggered User Action Control of AREyepiece Facility”).

Additional U.S. patent applications which appear to be mostappropriately classified into this category include: 20120194549(Osterhout et al., Aug. 2, 2012, “AR Glasses Specific User InterfaceBased on a Connected External Device Type”); 20120194550 (Osterhout etal., Aug. 2, 2012, “Sensor-Based Command and Control of External Deviceswith Feedback from the External Device to the AR Glasses”); 20120194551(Osterhout et al., Aug. 2, 2012, “AR Glasses with User-Action BasedCommand and Control of External Devices”); 20120194552 (Osterhout etal., Aug. 2, 2012, “AR Glasses with Predictive Control of ExternalDevice Based on Event Input”); 20120194553 (Osterhout et al., Aug. 2,2012, “AR Glasses with Sensor and User Action Based Control of ExternalDevices with Feedback”); 20120200488 (Osterhout et al., Aug. 9, 2012,“AR Glasses with Sensor and User Action Based Control of EyepieceApplications with Feedback”); 20120200499 (Osterhout et al., Aug. 9,2012, “AR Glasses with Event, Sensor, and User Action Based Control ofApplications Resident on External Devices with Feedback”); 20120200601(Osterhout et al., Aug. 9, 2012, “AR Glasses with State Triggered EyeControl Interaction with Advertising Facility”); 20120201725 (Imran,Aug. 9, 2012, “Micro-Fluidic Device”); 20120203081 (Leboeuf et al., Aug.9, 2012, “Physiological and Environmental Monitoring Apparatus andSystems”); 20120206322 (Osterhout et al., Aug. 16, 2012, “AR Glasseswith Event and Sensor Input Triggered User Action Capture Device Controlof AR Eyepiece Facility”); 20120206323 (Osterhout et al., Aug. 16, 2012,“AR Glasses with Event and Sensor Triggered AR Eyepiece Interface toExternal Devices”); and 20120206334 (Osterhout et al., Aug. 16, 2012,“AR Glasses with Event and User Action Capture Device Control ofExternal Applications”).

Further U.S. patent applications in this category include: 20120206335(Osterhout et al., Aug. 16, 2012, “AR Glasses with Event, Sensor, andUser Action Based Direct Control of External Devices with Feedback”);20120206485 (Osterhout et al., Aug. 16, 2012, “AR Glasses with Event andSensor Triggered User Movement Control of AR Eyepiece Facilities”);20120212398 (Border et al., Aug. 23, 2012, “See-Through Near-Eye DisplayGlasses Including a Partially Reflective, Partially Transmitting OpticalElement”); 20120212399 (Border et al., Aug. 23, 2012, “See-ThroughNear-Eye Display Glasses Wherein Image Light Is Transmitted to andReflected from an Optically Flat Film”); 20120212400 (Border et al.,Aug. 23, 2012, “See-Through Near-Eye Display Glasses Including a CurvedPolarizing Film in the Image Source, a Partially Reflective, PartiallyTransmitting Optical Element and an Optically Flat Film”); 20120212406(Osterhout et al., Aug. 23, 2012, “AR Glasses with Event and SensorTriggered AR Eyepiece Command and Control Facility of the AR Eyepiece”);20120212414 (Osterhout et al., Aug. 23, 2012, “AR Glasses with Event andSensor Triggered Control of AR Eyepiece Applications”); 20120218172(Border et al., Aug. 30, 2012, “See-Through Near-Eye Display Glasseswith a Small Scale Image Source”); 20120218301 (Miller, Aug. 30, 2012,“See-Through Display with an Optical Assembly Including a Wedge-ShapedIllumination System”); 20120235883 (Border et al., Sep. 20, 2012,“See-Through Near-Eye Display Glasses with a Light Transmissive WedgeShaped Illumination System”); 20120235885 (Miller et al., Sep. 20, 2012,“Grating in a Light Transmissive Illumination System for See-ThroughNear-Eye Display Glasses”); and 20120235886 (Border et al., Sep. 20,2012, “See-Through Near-Eye Display Glasses with a Small Scale ImageSource”).

Further U.S. patent applications in this category include: 20120235887(Border et al., Sep. 20, 2012, “See-Through Near-Eye Display GlassesIncluding a Partially Reflective, Partially Transmitting Optical Elementand an Optically Flat Film”); 20120235900 (Border et al., Sep. 20, 2012,“See-Through Near-Eye Display Glasses with a Fast Response PhotochromicFilm System for Quick Transition from Dark to Clear”); 20120236030(Border et al., Sep. 20, 2012, “See-Through Near-Eye Display GlassesIncluding a Modular Image Source”); 20120236031 (Haddick et al., Sep.20, 2012, “System and Method for Delivering Content to a Group ofSee-Through Near Eye Display Eyepieces”); 20120242678 (Border et al.,Sep. 27, 2012, “See-Through Near-Eye Display Glasses Including anAuto-Brightness Control for the Display Brightness Based on theBrightness in the Environment”); 20120242697 (Border et al., Sep. 27,2012, “See-Through Near-Eye Display Glasses with the Optical AssemblyIncluding Absorptive Polarizers or Anti-Reflective Coatings to ReduceStray Light”); 20120242698 (Haddick et al., Sep. 27, 2012, “See-ThroughNear-Eye Display Glasses with a Multi-Segment Processor-ControlledOptical Layer”); 20120249797 (Haddick et al., Oct. 4, 2012, “Head-WornAdaptive Display”); 20120302855 (Kamath et al., Nov. 29, 2012, “Systemsand Methods for Processing Sensor Data”); 20130035563 (Angelides, Feb.7, 2013, “Progressively Personalized Wireless-Based Interactive DiabetesTreatment”); 20130083003 (Perez et al., Apr. 4, 2013, “PersonalAudio/Visual System”); 20130083064 (Geisner et al., Apr. 4, 2013,“Personal Audio/Visual Apparatus Providing Resource Management”); and20130095459 (Tran, Apr. 18, 2013, “Health Monitoring System”). Prior artwhich appears to be most appropriately classified into this categoryalso includes: U.S. patent application Ser. No. 13/523,739 (Connor, Jun.14, 2012, “The Willpower Watch™: A Wearable Food Consumption Monitor”);U.S. patent application Ser. No. 13/616,238 (Connor, Sep. 14, 2012,“Interactive Voluntary and Involuntary Caloric Intake Monitor”); and WO2003032629 (Grosvenor, Apr. 17, 2003, “Automatic Photography”).

(4) Wearable Devices to Monitor Caloric Expenditure Activities and toHelp Measure Food Consumption

Wearable devices and methods in this category provide at least somemeasurement of both caloric expenditure activities and food consumption,but their measurement of food consumption is much less automated andaccurate than that of caloric expenditure activities. In some respects,devices and methods in this category are like those in the firstcategory, with the addition of caloric expenditure monitoring.

Most of the devices and methods in this category include a wearableaccelerometer (and possibly also other wearable sensors) for measuringcaloric expenditure, but rely on non-automated logging of foodconsumption information through a human-to-computer interface. Most ofthe devices and methods in this category display information concerningfood consumption as part of the energy balance equation, but do notautomatically collect this food consumption information.

Wearable devices and methods in this category are a useful step towarddeveloping wearable energy balance devices that can help people tomonitor and manage their energy balance and weight. However, prior artin this category has limitations with respect to the accuracy of foodconsumption measurement. These limitations are generally the same as thelimitations of devices and methods in the first category (non-wearabledevices to help measure food consumption). Their accuracy dependscritically on the consistency with which a person enters informationinto the device and the accuracy with which the person assesses theamounts and ingredients of non-standard foods consumed. Both of thesefactors can be problematic.

Specific examples of potentially-relevant prior art which appear to bemost appropriately classified into this category include the followingU.S. patents: U.S. Pat. No. 6,095,949 (Arai, Aug. 1, 2000, “HealthManagement Device”); U.S. Pat. No. 6,506,152 (Lackey et al., Jan. 14,2003, “Caloric Energy Balance Monitor”); U.S. Pat. No. 6,571,200 (Mault,May 27, 2003, “Monitoring Caloric Expenditure Resulting from BodyActivity”); U.S. Pat. No. 6,635,015 (Sagel, Oct. 21, 2003, “Body WeightManagement System”); U.S. Pat. No. 6,675,041 (Dickinson, Jan. 6, 2004,“Electronic Apparatus and Method for Monitoring Net Calorie Intake”);U.S. Pat. No. 7,361,141 (Nissila et al., Apr. 22, 2008, “Method andDevice for Weight Management of Humans”); U.S. Pat. No. 8,180,591 (Yuenet al., May 15, 2012, “Portable Monitoring Devices and Methods ofOperating Same”); U.S. Pat. No. 8,180,592 (Yuen et al., May 15, 2012,“Portable Monitoring Devices and Methods of Operating Same”); U.S. Pat.No. 8,311,769 (Yuen et al., Nov. 13, 2012, “Portable Monitoring Devicesand Methods of Operating Same”); U.S. Pat. No. 8,311,770 (Yuen et al.,Nov. 13, 2012, “Portable Monitoring Devices and Methods of OperatingSame”); U.S. Pat. No. 8,386,008 (Yuen et al., Feb. 26, 2013, “ActivityMonitoring Systems and Methods of Operating Same”); U.S. Pat. No.8,386,008 (Yuen et al., Feb. 26, 2013, “Portable Monitoring Devices andMethods of Operating Same”); and U.S. Pat. No. 8,437,980 (Yuen et al.,May 7, 2013, “Portable Monitoring Devices and Methods of OperatingSame”).

Specific examples of potentially-relevant prior art which appear to bemost appropriately classified into this category also include thefollowing U.S. patent applications: 20020109600 (Mault et al., Aug. 15,2002, “Body Supported Activity and Condition Monitor”); 20020156351(Sagel, Oct. 24, 2002, “Body Weight Management System”); 20050004436(Nissila et al., Jan. 6, 2005, “Method and Device for Weight Managementof Humans”); 20100079291 (Kroll et al., Apr. 1, 2010, “PersonalizedActivity Monitor and Weight Management System”); 20100228160 (Schweizer,Sep. 9, 2010, “Apparatus for Activity Monitoring”); 20110087137 (Hanoun,Apr. 14, 2011, “Mobile Fitness and Personal Caloric Management System”);20120083705 (Yuen et al., Apr. 5, 2012, “Activity Monitoring Systems andMethods of Operating Same”); 20120083714 (Yuen et al., Apr. 5, 2012,“Activity Monitoring Systems and Methods of Operating Same”);20120083715 (Yuen et al., Apr. 5, 2012, “Portable Monitoring Devices andMethods of Operating Same”); 20120083716 (Yuen et al., Apr. 5, 2012,“Portable Monitoring Devices and Methods of Operating Same”);20120084053 (Yuen et al., Apr. 5, 2012, “Portable Monitoring Devices andMethods of Operating Same”); 20120084054 (Yuen et al., Apr. 5, 2012,“Portable Monitoring Devices and Methods of Operating Same”); and20120226471 (Yuen et al., Sep. 6, 2012, “Portable Monitoring Devices andMethods of Operating Same”).

Additional U.S. patent applications which appear to be mostappropriately classified into this category include: 20120226472 (Yuenet al., Sep. 6, 2012, “Portable Monitoring Devices and Methods ofOperating Same”); 20120316458 (Rahman et al., Dec. 13, 2012,“Data-Capable Band for Medical Diagnosis, Monitoring, and Treatment”);20120316896 (Rahman et al., Dec. 13, 2012, “Personal Advisor SystemUsing Data-Capable Band”); 20120316932 (Rahman et al., Dec. 13, 2012,“Wellness Application for Data-Capable Band”); 20120316932 (Rahman etal., Dec. 13, 2012, “Wellness Application for Data-Capable Band”);20120317167 (Rahman et al., Dec. 13, 2012, “Wellness Application forData-Capable Band”); 20130006125 (Kroll et al., Jan. 3, 2013,“Personalized Activity Monitor and Weight Management System”);20130029807 (Amsel, Jan. 31, 2013, “Health Tracking Program”);20130073254 (Yuen et al., Mar. 21, 2013, “Portable Monitoring Devicesand Methods of Operating Same”); 20130073255 (Yuen et al., Mar. 21,2013, “Portable Monitoring Devices and Methods of Operating Same”);20130080113 (Yuen et al., Mar. 28, 2013, “Portable Monitoring Devicesand Methods of Operating Same”); and 20130096843 (Yuen et al., Apr. 18,2013, “Portable Monitoring Devices and Methods of Operating Same”).

(5) Wearable Devices to Monitor and Measure Both Caloric ExpenditureActivities and Food Consumption

Wearable devices and methods in this category provide monitoring andmeasurement of both caloric expenditure activities and food consumption.Their monitoring and measurement of food consumption is generally not asautomated or accurate as the monitoring and measurement of caloricexpenditure activities, but devices in this category are a significantstep toward integrated wearable energy balance devices. In somerespects, devices and methods in this category are like those in thethird category, with the addition of caloric expenditure monitoring.

Although wearable device and methods in this category are a significantstep toward developing integrated energy balance devices which can beuseful for energy balance, weight management, and proper nutrition,prior art in this category has not yet solved the dilemma of personalprivacy vs. accuracy of food consumption measurement. Some prior art inthis category offers relatively-low privacy intrusion, but hasrelatively-low accuracy of food consumption measurement. Other prior artin this category offers relatively-high accuracy for food consumptionmeasurement, but comes with relatively-high privacy intrusion. Theinvention that we will disclose later will solve this problem byoffering relatively-high accuracy for food consumption measurement withrelatively-low privacy intrusion.

Specific examples of potentially-relevant prior art which appear to bemost appropriately classified into this category include the followingU.S. patents: U.S. Pat. No. 6,513,532 (Mault et al., Feb. 4, 2003, “Dietand Activity Monitoring Device”); U.S. Pat. No. 6,605,038 (Teller etal., Aug. 12, 2003, “System for Monitoring Health, Wellness andFitness”); U.S. Pat. No. 6,790,178 (Mault et al., Sep. 14, 2004,“Physiological Monitor and Associated Computation, Display andCommunication Unit”); U.S. Pat. No. 7,020,508 (Stivoric et al., Mar. 28,2006, “Apparatus for Detecting Human Physiological and ContextualInformation”); U.S. Pat. No. 7,261,690 (Teller et al., Aug. 28, 2007,“Apparatus for Monitoring Health, Wellness and Fitness”); U.S. Pat. No.7,285,090 (Stivoric et al., Oct. 23, 2007, “Apparatus for Detecting,Receiving, Deriving and Displaying Human Physiological and ContextualInformation”); U.S. Pat. No. 7,689,437 (Teller et al., Mar. 30, 2010,“System for Monitoring Health, Wellness and Fitness”); U.S. Pat. No.7,914,468 (Shalon et al., Mar. 29, 2011, “Systems and Methods forMonitoring and Modifying Behavior”); U.S. Pat. No. 7,959,567 (Stivoricet al., Jun. 14, 2011, “Device to Enable Quick Entry of CaloricContent”); U.S. Pat. No. 8,073,707 (Teller et al., Dec. 6, 2011, “Systemfor Detecting Monitoring and Reporting an Individual's Physiological orContextual Status”); U.S. Pat. No. 8,157,731 (Teller et al., Apr. 17,2012, “Method and Apparatus for Auto Journaling of Continuous orDiscrete Body States Utilizing Physiological and/or ContextualParameters”); U.S. Pat. No. 8,323,189 (Tran et al., Dec. 4, 2012,“Health monitoring appliance”); U.S. Pat. No. 8,328,718 (Tran, Dec. 11,2012, “Health Monitoring Appliance”); U.S. Pat. No. 8,398,546 (Pacioneet al., Mar. 19, 2013, “System for Monitoring and Managing Body Weightand Other Physiological Conditions Including Iterative and PersonalizedPlanning, Intervention and Reporting Capability”); and U.S. Pat. No.8,425,415 (Tran, Apr. 23, 2013, “Health Monitoring Appliance”).

Specific examples of potentially-relevant prior art which appear to bemost appropriately classified into this category also include thefollowing U.S. patent applications: 20010049470 (Mault et al., Dec. 6,2001, “Diet and Activity Monitoring Device”); 20020027164 (Mault et al.,Mar. 7, 2002, “Portable Computing Apparatus Particularly Useful in aWeight Management Program”); 20020047867 (Mault et al., Apr. 25, 2002,“Image Based Diet Logging”); 20020133378 (Mault et al., Sep. 19, 2002,“System and Method of Integrated Calorie Management”); 20030065257(Mault et al., Apr. 3, 2003, “Diet and Activity Monitoring Device”);20030208110 (Mault et al., Nov. 6, 2003, “Physiological Monitoring usingWrist-Mounted Device”); 20040034289 (Teller et al., Feb. 19, 2004,“System for Monitoring Health, Wellness and Fitness”); 20040133081(Teller et al., Jul. 8, 2004, “Method and Apparatus for Auto Journalingof Continuous or Discrete Body States Utilizing Physiological and/orContextual Parameters”); 20040152957 (Stivoric et al., Aug. 5, 2004,“Apparatus for Detecting, Receiving, Deriving and Displaying HumanPhysiological and Contextual Information”); 20050113650 (Pacione et al.,May 26, 2005, “System for Monitoring and Managing Body Weight and OtherPhysiological Conditions Including Iterative and Personalized PlanningIntervention and Reporting Capability”); 20060031102 (Teller et al.,Feb. 9, 2006, “System for Detecting Monitoring and Reporting anIndividual's Physiological or Contextual Status”); 20060064037 (Shalonet al., Mar. 23, 2006, “Systems and Methods for Monitoring and ModifyingBehavior”); 20060122474 (Teller et al., Jun. 8, 2006, “Apparatus forMonitoring Health Wellness and Fitness”); and 20060264730 (Stivoric etal., Nov. 23, 2006, “Apparatus for Detecting Human Physiological andContextual Information”).

Additional U.S. patent applications which appear to be mostappropriately classified into this category include: 20070100666(Stivoric et al., May 3, 2007, “Devices and Systems for Contextual andPhysiological-Based Detection, Monitoring, Reporting, Entertainment, andControl of Other Devices”); 20080161654 (Teller et al., Jul. 3, 2008,“Method and Apparatus for Auto Journaling of Body States and ProvidingDerived Physiological States Utilizing Physiological and/or ContextualParameter”); 20080161655 (Teller et al., Jul. 3, 2008, “Method andApparatus for Auto Journaling of Body States and Providing DerivedPhysiological States Utilizing Physiological and/or ContextualParameter”); 20080167535 (Andre et. al, Jul. 10, 2008, “Devices andSystems for Contextual and Physiological-Based Reporting, Entertainment,Control of Other Devices, Health Assessment and Therapy”); 20080167536(Teller et al., Jul. 10, 2008, “Method and Apparatus for Auto Journalingof Body States and Providing Derived Physiological States UtilizingPhysiological and/or Contextual Parameter”); 20080167537 (Teller et al.,Jul. 10, 2008, “Method and Apparatus for Auto Journaling of Body Statesand Providing Derived Physiological States Utilizing Physiologicaland/or Contextual Parameter”); 20080167538 (Teller et al., Jul. 10,2008, “Method and Apparatus for Auto Journaling of Body States andProviding Derived Physiological States Utilizing Physiological and/orContextual Parameter”); 20080167539 (Teller et al., Jul. 10, 2008,“Method and Apparatus for Auto Journaling of Body States and ProvidingDerived Physiological States Utilizing Physiological and/or ContextualParameter”); and 20080171920 (Teller et al., Jul. 17, 2008, “Method andApparatus for Auto Journaling of Body States and Providing DerivedPhysiological States Utilizing Physiological and/or ContextualParameter”).

Further U.S. patent applications in this category include: 20080171921(Teller et al., Jul. 17, 2008, “Method and Apparatus for Auto Journalingof Body States and Providing Derived Physiological States UtilizingPhysiological and/or Contextual Parameter”); 20080171922 (Teller et al.,Jul. 17, 2008, “Method and Apparatus for Auto Journaling of Body Statesand Providing Derived Physiological States Utilizing Physiologicaland/or Contextual Parameter”); 20080275309 (Stivoric et al., Nov. 6,2008, “Input Output Device for Use with Body Monitor”); 20090012433(Fernstrom et al., Jan. 8, 2009, “Method, Apparatus and System for FoodIntake and Physical Activity Assessment”); 20090177068 (Stivoric et al.,Jul. 9, 2009, “Method and Apparatus for Providing Derived GlucoseInformation Utilizing Physiological and/or Contextual Parameters”);20110125063 (Shalon et al., May 26, 2011, “Systems and Methods forMonitoring and Modifying Behavior”); 20110276312 (Shalon et al., Nov.10, 2011, “Device for Monitoring and Modifying Eating Behavior”);20120313776 (Utter et al., Dec. 13, 2012, “General Health and WellnessManagement Method and Apparatus for a Wellness Application Using Datafrom a Data-Capable Band”); 20120313776 (Utter, Dec. 13, 2012, “GeneralHealth and Wellness Management Method and Apparatus for a WellnessApplication Using Data from a Data-Capable Band”); and 20120326873(Utter, Dec. 27, 2012, “Activity Attainment Method and Apparatus for aWellness Application Using Data from a Data-Capable Band”).

Further U.S. patent applications in this category include: 20120326873(Utter, Dec. 27, 2012, “Activity Attainment Method and Apparatus for aWellness Application Using Data from a Data-Capable Band”); 20120330109(Tran, Dec. 27, 2012, “Health Monitoring Appliance”); 20130002435(Utter, Jan. 3, 2013, “Sleep Management Method and Apparatus for aWellness Application Using Data from a Data-Capable Band”); 20130004923(Utter, Jan. 3, 2013, “Nutrition Management Method and Apparatus for aWellness Application Using Data from a Data-Capable Band”); 20130069780(Tran et al., Mar. 21, 2013, “Health Monitoring Appliance”); and20130072765 (Kahn et al., Mar. 21, 2013, “Body-Worn Monitor”). Prior artwhich appears to be most appropriately classified into this categoryalso includes: WO 2005029242 (Pacione et al., Jun. 9, 2005, “System forMonitoring and Managing Body Weight and Other Physiological ConditionsIncluding Iterative and Personalized Planning, Intervention andReporting Capability”); WO 2010070645 (Einav, Jun. 24, 2010, “Method andSystem for Monitoring Eating Habits”); and WO 2012170584 (Utter, Dec.13, 2012, “General Health and Wellness Management Method and Apparatusfor a Wellness Application Using Data from a Data-Capable Band”).

(6) Other Potentially-Relevant Devices and Methods

When reviewing the prior art, I found a number of examples of prior artthat may be potentially relevant to this present invention but which donot fall neatly into one of the above five categories. I include themhere in a miscellaneous category of other potentially-relevant devicesand methods. The titles are given to help the reader get insights intotheir diverse, but potentially-relevant, contents. Specific examples ofpotentially-relevant prior art which appear to be most appropriatelyclassified into this category include the following U.S. patents: U.S.Pat. No. 3,885,576 (Symmes, May 27, 1975, “Wrist Band Including aMercury Switch to Induce an Electric Shock”); U.S. Pat. No. 4,221,959(Sessler, Sep. 9, 1980, “Checking Device for Checking the Food Intake”);U.S. Pat. No. 4,310,316 (Thomann, Jan. 12, 1982, “Diet ControlApparatus”); U.S. Pat. No. 4,355,645 (Mitani et al., Oct. 26, 1982,“Device for Displaying Masticatory Muscle Activities”); U.S. Pat. No.4,819,860 (Hargrove et al., Apr. 11, 1989, “Wrist-Mounted VitalFunctions Monitor and Emergency Locator”); U.S. Pat. No. 4,917,108(Mault, Apr. 17, 1990, “Oxygen Consumption Meter”); U.S. Pat. No.5,148,002 (Kuo et al., Sep. 15, 1992, “Multi-Functional GarmentSystem”); U.S. Pat. No. 5,285,398 (Janik, Feb. 8, 1994, “FlexibleWearable Computer”); U.S. Pat. No. 5,301,679 (Taylor, Apr. 12, 1994,“Method and System for Analysis of Body Sounds”); U.S. Pat. No.5,491,651 (Janik, Feb. 13, 1996, “Flexible Wearable Computer”); U.S.Pat. No. 5,515,858 (Myllymaki, May 14, 1996, “Wrist-Held MonitoringDevice for Physical Condition”); U.S. Pat. No. 5,555,490 (Carroll, Sep.10, 1996, “Wearable Personal Computer System”); U.S. Pat. No. 5,581,492(Janik, Dec. 3, 1996, “Flexible Wearable Computer”); U.S. Pat. No.5,636,146 (Flentov et al., Jun. 3, 1997, “Apparatus and Methods forDetermining Loft Time and Speed”); U.S. Pat. No. 5,908,301 (Lutz, Jun.1, 1999, “Method and Device for Modifying Behavior”); U.S. Pat. No.6,095,985 (Raymond et al., Aug. 1, 2000, “Health Monitoring System”);U.S. Pat. No. 6,218,358 (Firestein et al., Apr. 17, 2001, “FunctionalExpression of, and Assay for, Functional Cellular Receptors In Vivo”);U.S. Pat. No. 6,266,623 (Vock et al., Jul. 24, 2001, “Sport MonitoringApparatus for Determining Loft Time, Speed, Power Absorbed and OtherFactors Such as Height”); U.S. Pat. No. 6,387,329 (Lewis et al., May 14,2002, “Use of an Array of Polymeric Sensors of Varying Thickness forDetecting Analytes in Fluids”); U.S. Pat. No. 6,473,368 (Stanfield, Oct.29, 2002, “Consumption Controller”); and U.S. Pat. No. 6,572,542 (Houbenet al., Jun. 3, 2003, “System and Method for Monitoring and Controllingthe Glycemic State of a Patient”).

Additional U.S. patents which appear to be most appropriately classifiedinto this category include: U.S. Pat. No. 6,595,929 (Stivoric et al.,Jul. 22, 2003, “System for Monitoring Health Wellness and Fitness Havinga Method and Apparatus for Improved Measurement of Heat Flow”); U.S.Pat. No. 6,610,367 (Lewis et al., Aug. 26, 2003, “Use of an Array ofPolymeric Sensors of Varying Thickness for Detecting Analytes inFluids”); U.S. Pat. No. 6,765,488 (Stanfield, Jul. 20, 2004, “EnhancedConsumption Controller”); U.S. Pat. No. 6,850,861 (Faiola et al., Feb.1, 2005, “System for Monitoring Sensing Device Data Such as Food SensingDevice Data”); U.S. Pat. No. 7,122,152 (Lewis et al., Oct. 17, 2006,“Spatiotemporal and Geometric Optimization of Sensor Arrays forDetecting Analytes Fluids”); U.S. Pat. No. 7,192,136 (Howell et al.,Mar. 20, 2007, “Tethered Electrical Components for Eyeglasses”); U.S.Pat. No. 7,241,880 (Adler et al., Jul. 10, 2007, “T1R Taste Receptorsand Genes Encoding Same”); U.S. Pat. No. 7,247,023 (Peplinski et al.,Jul. 24, 2007, “System and Method for Monitoring Weight and Nutrition”);U.S. Pat. No. 7,502,643 (Farringdon et al., Mar. 10, 2009, “Method andApparatus for Measuring Heart Related Parameters”); U.S. Pat. No.7,595,023 (Lewis et al., Sep. 29, 2009, “Spatiotemporal and GeometricOptimization of Sensor Arrays for Detecting Analytes in Fluids”); U.S.Pat. No. 7,651,868 (Mcdevitt et al., Jan. 26, 2010, “Method and Systemfor the Analysis of Saliva using a Sensor Array”); U.S. Pat. No.7,882,150 (Badyal, Feb. 1, 2011, “Health Advisor”); U.S. Pat. No.7,905,815 (Ellis et al., Mar. 15, 2011, “Personal Data CollectionSystems and Methods”); U.S. Pat. No. 7,905,832 (Lau et al., Mar. 15,2011, “Method and System for Personalized Medical Monitoring andNotifications Therefor”); and U.S. Pat. No. 7,931,562 (Ellis et al.,Apr. 26, 2011, “Mobile Data Logging Systems and Methods”).

Further U.S. patents in this category include: U.S. Pat. No. 8,067,185(Zoller et al., Nov. 29, 2011, “Methods of Quantifying Taste ofCompounds for Food or Beverages”); U.S. Pat. No. 8,116,841 (Bly et al.,Feb. 14, 2012, “Adherent Device with Multiple Physiological Sensors”);U.S. Pat. No. 8,121,673 (Tran, Feb. 12, 2012, “Health MonitoringAppliance”); U.S. Pat. No. 8,170,656 (Tan et al., May 1, 2012, “WearableElectromyography-Based Controllers for Human-Computer Interface”); U.S.Pat. No. 8,275,635 (Stivoric et al., Sep. 25, 2012, “Integration ofLifeotypes with Devices and Systems”); U.S. Pat. No. 8,285,356 (Bly etal., Oct. 9, 2012, “Adherent Device with Multiple PhysiologicalSensors”); U.S. Pat. No. 8,314,224 (Adler et al., Nov. 20, 2012, “T1RTaste Receptors and Genes Encoding Same”); U.S. Pat. No. 8,323,188(Tran, Dec. 4, 2012, “Health Monitoring Appliance”); U.S. Pat. No.8,323,218 (Davis et al., Dec. 4, 2012, “Generation of ProportionalPosture Information Over Multiple Time Intervals”); U.S. Pat. No.8,334,367 (Adler, Dec. 18, 2012, “T2R Taste Receptors and Genes EncodingSame”); U.S. Pat. No. 8,340,754 (Chamney et al., Dec. 25, 2012, “Methodand a Device for Determining the Hydration and/or Nutrition Status of aPatient”); U.S. Pat. No. 8,344,325 (Merrell et al., Jan. 1, 2013,“Electronic Device With Sensing Assembly and Method for Detecting BasicGestures”); U.S. Pat. No. 8,344,998 (Fitzgerald et al., Jan. 1, 2013,“Gesture-Based Power Management of a Wearable Portable Electronic Devicewith Display”); U.S. Pat. No. 8,345,414 (Mooring et al., Jan. 1, 2013,“Wearable Computing Module”); U.S. Pat. No. 8,364,250 (Moon et al., Jan.29, 2013, “Body-Worn Vital Sign Monitor”); and U.S. Pat. No. 8,369,936(Farringdon et al., Feb. 5, 2013, “Wearable Apparatus for MeasuringHeart-Related Parameters and Deriving Human Status Parameters fromSensed Physiological and Contextual Parameters”).

Further U.S. patents in this category include: U.S. Pat. No. 8,370,176(Vespasiani, Feb. 5, 2013, “Method and System for Defining andInteractively Managing a Watched Diet”); U.S. Pat. No. 8,379,488(Gossweiler et al., Feb. 19, 2013, “Smart-Watch Including Flip UpDisplay”); U.S. Pat. No. 8,382,482 (Miller-Kovach et al., Feb. 26, 2013,“Processes and Systems for Achieving and Assisting in Improved NutritionBased on Food Energy Data and Relative Healthfulness Data”); U.S. Pat.No. 8,382,681 (Escutia et al., Feb. 26, 2013, “Fully Integrated Wearableor Handheld Monitor”); U.S. Pat. No. 8,409,118 (Agrawal et al., Apr. 2,2013, “Upper Arm Wearable Exoskeleton”); U.S. Pat. No. 8,417,298(Mittleman et al., Apr. 9, 2013, “Mounting Structures for PortableElectronic Devices”); U.S. Pat. No. 8,419,268 (Yu Apr. 16, 2013,“Wearable Electronic Device”); U.S. Pat. No. 8,421,634 (Tan et al., Apr.16, 2013, “Sensing Mechanical Energy to Appropriate the Body for DataInput”); U.S. Pat. No. 8,423,378 (Goldberg, Apr. 16, 2013, “FacilitatingHealth Care Management of Subjects”); U.S. Pat. No. 8,423,380 (GellyApr. 16, 2013, “Method and System for Interactive Health RegimenAccountability and Patient Monitoring”); and U.S. Pat. No. 8,437,823(Ozawa et al., May 7, 2013, “Noninvasive Living Body MeasurementApparatus and Noninvasive Living Body Measurement Method”).

Specific examples of potentially-relevant prior art which appear to bemost appropriately classified into this category also include thefollowing U.S. patent applications: 20020049482 (Fabian et al., Apr. 25,2002, “Lifestyle Management System”); 20040100376 (Lye et al., May 27,2004, “Healthcare Monitoring System”); 20050113649 (Bergantino, May 26,2005, “Method and Apparatus for Managing a User's Health”); 20050146419(Porter, Jul. 7, 2005, “Programmable Restricted Access Food StorageContainer and Behavior Modification Assistant”); 20050263160 (Utley etal., Dec. 1, 2005, “Intraoral Aversion Devices and Methods”);20060015016 (Thornton, Jan. 19, 2006, “Caloric Balance Weight ControlSystem and Methods of Making and Using Same”); 20060122468 (Tavor, Jun.8, 2006, “Nutritional Counseling Method and Server”); 20070106129(Srivathsa et al., May 10, 2007, “Dietary Monitoring System forComprehensive Patient Management”); 20080036737 (Hernandez-Rebollar,Feb. 14, 2008, “Arm Skeleton for Capturing Arm Position and Movement”);20080140444 (Karkanias et al., Jun. 12, 2008, “Patient Monitoring ViaImage Capture”); 20090261987 (Sun, Oct. 22, 2009, “Sensor InstrumentSystem Including Method for Detecting Analytes in Fluids”); 20100000292(Karabacak et al., Jan. 7, 2010, “Sensing Device”); 20100049004 (Edmanet al., Feb. 25, 2010, “Metabolic Energy Monitoring System”);20100049010 (Goldreich, Feb. 25, 2010, “Method and Device for MeasuringPhysiological Parameters at the Wrist”); 20100055271 (Miller-Kovach etal., Mar. 4, 2010, “Processes and Systems Based on Metabolic ConversionEfficiency”); 20100055652 (Miller-Kovach et al., Mar. 4, 2010,“Processes and Systems Based on Dietary Fiber as Energy”); and20100055653 (Miller-Kovach et al., Mar. 4, 2010, “Processes and SystemsUsing and Producing Food Healthfulness Data Based on Food Metagroups”).

Additional U.S. patent applications which appear to be mostappropriately classified into this category include: 20100209897 (Utleyet al., Aug. 19, 2010, “Intraoral Behavior Monitoring and AversionDevices and Methods”); 20100291515 (Pinnisi et al., Nov. 18, 2010,“Regulating Food and Beverage Intake”); 20110053128 (Alman, Mar. 3,2011, “Automated Patient Monitoring and Counseling System”); 20110077471(King, Mar. 31, 2011, “Treatment and Prevention of Overweight andObesity by Altering Visual Perception of Food During Consumption”);20110205851 (Harris, Aug. 25, 2011, “E-Watch”); 20110218407 (Haberman etal., Sep. 8, 2011, “Method and Apparatus to Monitor, Analyze andOptimize Physiological State of Nutrition”); 20120015432 (Adler, Jan.19, 2012, “Isolated Bitter Taste Receptor Polypeptides”); 20120021388(Arbuckle et al., Jan. 26, 2012, “System and Method for WeightManagement”); 20120053426 (Webster et al., Mar. 1, 2012, “System andMethod for Measuring Calorie Content of a Food Sample”); 20120071731(Gottesman, Mar. 22, 2012, “System and Method for PhysiologicalMonitoring”); 20120179020 (Wekell, Jul. 12, 2012, “Patient MonitoringDevice”); 20120188158 (Tan et al., Jul. 26, 2012, “WearableElectromyography-Based Human-Computer Interface”); 20120214594 (Kirovskiet al., Aug. 23, 2012, “Motion Recognition”); 20120231960 (Osterfeld etal., Sep. 13, 2012, “Systems and Methods for High-Throughput Detectionof an Analyte in a Sample”); 20120235647 (Chung et al., Sep. 20, 2012,“Sensor with Energy-Harvesting Device”); 20120239304 (Hayter et al.,Sep. 20, 2012, “Method and System for Determining Analyte Levels”);20120242626 (Hu, Sep. 27, 2012, “Electronic Watch Capable of AdjustingDisplay Angle of Screen Content Thereof”); and 20120245472 (Rulkov etal., Sep. 27, 2012, “Monitoring Device with an Accelerometer, Method andSystem”).

Further U.S. patent applications in this category include: 20120245714(Mueller et al., Sep. 27, 2012, “System and Method for Counting SwimmingLaps”); 20120254749 (Downs et al., Oct. 4, 2012, “System and Method forControlling Life Goals”); 20120258804 (Ahmed, Oct. 11, 2012,“Motion-Based Input for Platforms and Applications”); 20120277638(Skelton et al., Nov. 1, 2012, “Obtaining Baseline PatientInformation”); 20120303638 (Bousamra et al., Nov. 29, 2012, “LocationEnabled Food Database”); 20120315986 (Walling, Dec. 13, 2012, “VirtualPerformance System”); 20120316793 (Jung et al., Dec. 13, 2012, “Methodsand Systems for Indicating Behavior in a Population Cohort”);20120326863 (Johnson et al., Dec. 27, 2012, “Wearable Portable Deviceand Method”); 20120330112 (Lamego et al., Dec. 27, 2012, “PatientMonitoring System”); 20120331201 (Rondel, Dec. 27, 2012, “Strap-BasedComputing Device”); 20130002538 (Mooring et al., Jan. 3, 2013,“Gesture-Based User Interface for a Wearable Portable Device”);20130002545 (Heinrich et al., Jan. 3, 2013, “Wearable Computer withCurved Display and Navigation Tool”); 20130002724 (Heinrich et al., Jan.3, 2013, “Wearable Computer with Curved Display and Navigation Tool”);20130009783 (Tran, Jan. 10, 2013, “Personal Emergency Response (PER)System”); 20130017789 (Chi et al., Jan. 17, 2013, “Systems and Methodsfor Accessing an Interaction State Between Multiple Devices”);20130021226 (Bell, Jan. 24, 2013, “Wearable Display Devices”);20130021658 (Miao et al., Jan. 24, 2013, “Compact See-Through DisplaySystem”); 20130027060 (Tralshawala et al., Jan. 31, 2013, “Systems andMethods for Non-Destructively Measuring Calorie Contents of FoodItems”); and 20130035575 (Mayou et al., Feb. 7, 2013, “Systems andMethods for Detecting Glucose Level Data Patterns”).

Further U.S. patent applications in this category include: 20130035865(Mayou et al., Feb. 7, 2013, “Systems and Methods for Detecting GlucoseLevel Data Patterns”); 20130038056 (Donelan et al., Feb. 14, 2013,“Methods and Apparatus for Harvesting Biomechanical Energy”);20130041272 (Guillen et al., Feb. 14, 2013, “Sensor Apparatus Adapted tobe Incorporated in a Garment”); 20130044042 (Olsson et al., Feb. 21,2013, “Wearable Device with Input and Output Structures”); 20130045037(Schaffer, Feb. 21, 2013, “Wristwatch Keyboard”); 20130048737 (Baym etal., Feb. 28, 2013, “Systems, Devices, Admixtures, and Methods IncludingTransponders for Indication of Food Attributes”); 20130048738 (Baym etal., Feb. 28, 2013, “Systems, Devices, Admixtures, and Methods IncludingTransponders for Indication of Food Attributes”); 20130049931 (Baym etal., Feb. 28, 2013, “Systems, Devices, Methods, and Admixtures ofTransponders and Food Products for Indication of Food Attributes”);20130049932 (Baym et al., Feb. 28, 2013, “Systems, Devices, Methods, andAdmixtures of Transponders and Food Products for Indication of FoodAttributes”); 20130049933 (Baym et al., Feb. 28, 2013, “Systems,Devices, Methods, and Admixtures Including Interrogators andInterrogation of Tags for Indication of Food Attributes”); 20130049934(Baym et al., Feb. 28, 2013, “Systems, Devices, Methods, and AdmixturesIncluding Interrogators and Interrogation of Tags for Indication of FoodAttributes”); and 20130053655 (Castellanos, Feb. 28, 2013, “MobileVascular Health Evaluation Devices”).

Further U.S. patent applications in this category include: 20130053661(Alberth et al., Feb. 28, 2013, “System for Enabling Reliable SkinContract of an Electrical Wearable Device”); 20130063342 (Chen et al.,Mar. 14, 2013, “Human Interface Input Acceleration System”); 20130065680(Zavadsky et al., Mar. 14, 2013, “Method and Apparatus for FacilitatingStrength Training”); 20130069931 (Wilson et al., Mar. 21, 2013,“Correlating Movement Information Received from Different Sources”);20130069985 (Wong et al., Mar. 21, 2013, “Wearable Computer withSuperimposed Controls and Instructions for External Device”);20130070338 (Gupta et al., Mar. 21, 2013, “Lightweight Eyepiece for HeadMounted Display”); 20130072807 (Tran, Mar. 21, 2013, “Health MonitoringAppliance”); 20130083496 (Franklin et al., Apr. 4, 2013, “FlexibleElectronic Devices”); 20130100027 (Wang et al., Apr. 25, 2013, “PortableElectronic Device”); 20130107674 (Gossweiler et al., May 2, 2013,“Smart-Watch with User Interface Features”); 20130109947 (Wood, May 2,2013, “Methods and Systems for Continuous Non-Invasive Blood PressureMeasurement Using Photoacoustics”); 20130110549 (Lawn et al., May 2,2013, “Device and Method for Assessing Blood Glucose Control”);20130111611 (Barros Almedo et al., May 2, 2013, “Method to Measure theMetabolic Rate or Rate of Glucose Consumption of Cells or Tissues withHigh Spatiotemporal Resolution Using a Glucose Nanosensor”); 20130115717(Guo et al., May 9, 2013, “Analyzing Chemical and Biological SubstancesUsing Nano-Structure Based Spectral Sensing”); 20130116525 (Heller etal., May 9, 2013, “Analyte Monitoring Device and Methods of Use”);20130117040 (James et al., May 9, 2013, “Method and System forSupporting a Health Regimen”); 20130117041 (Boyce et al., May 9, 2013,“Computer Method and System for Promoting Health, Wellness, and Fitnesswith Multiple Sponsors”); 20130117135 (Riddiford et al., May 9, 2013,“Multi-User Food and Drink Ordering System”); and 20130119255 (Dickinsonet al., May 16, 2013, “Methods and Devices for Clothing Detection abouta Wearable Electronic Device”).

SUMMARY AND ADVANTAGES OF THIS INVENTION

This invention is a device, system, and method for monitoring foodconsumption comprising: (a) a wearable sensor that is configured to beworn on a person's body or clothing, wherein this wearable sensorautomatically collects data that is used to detect probable eatingevents without requiring action by the person in association with aprobable eating event apart from the act of eating, and wherein aprobable eating event is a period of time during which the person isprobably eating; (b) an imaging member, wherein this imaging member isused by the person to take pictures of food that the person eats,wherein using this imaging member to take pictures of food requiresvoluntary action by the person apart from the act of eating, and whereinthe person is prompted to take pictures of food using this imagingmember when data collected by the wearable sensor indicates a probableeating event; and (c) a data analysis component, wherein this componentanalyzes pictures of food taken by the imaging member to estimate thetypes and amounts of foods, ingredients, nutrients, and/or calories thatare consumed by the person.

In an example, the wearable sensor of this invention can be part of asmart watch or smart bracelet that is worn on a person's wrist andmonitors the person's wrist or hand motions to detect probable eatingevents. In an example, the imaging member of this invention can be asmart phone that the person is prompted to use to take pictures of foodduring a selected period of time associated with an eating event.

In the design of devices and systems for monitoring a person's foodconsumption, there can be a tradeoff between greater accuracy of foodconsumption measurement versus preservation of a person's privacy. Onthe one hand, it is possible to create a wearable device with highcompliance and accuracy for monitoring a person's food consumption, butsuch a device can be highly intrusive with respect the person's privacy.For example, one can create a wearable video camera that a person wearsall the time. This wearable camera can continually record video picturesof what the person does and/or sees in order to detect and measure foodconsumption. However, such a continuously-recording wearable camera canbe highly intrusive with respect to the person's privacy.

On the other hand, it is possible to create a system for measuring foodconsumption that relies only on a person's voluntary use of a hand-helddevice to take pictures of food, wherein these pictures are thenanalyzed to identify foods consumed. This approach can be good forpreserving a person's privacy, but can have low compliance for measuringtotal food consumption. People can easily forget, or otherwise fail, toconsistently take pictures of every meal and snack that they consume.Accordingly, there can be large gaps in the measurement of foodconsumption with this approach.

The invention disclosed herein addresses this tradeoff of accuracyversus privacy by integrating the operation of a wearable sensor (suchas a smart watch that automatically detects food events) and an imagingmember (such as a smart phone that the person is prompted to use to takepictures of food that they eat) to achieve relatively-high measurementaccuracy with relatively-low privacy intrusion. The strength of theprompt to take food pictures during eating events can be adjusteddepending on how strongly the person feels about the need forself-control and accurate measurement of food consumption. Thisinvention is a significant improvement over prior art that is based on asmart watch alone or a smart phone alone.

Information from this invention can be combined with a computer-to-humaninterface that provides feedback to encourage the person to eat healthyfoods and to limit excess consumption of unhealthy foods. In order to bereally useful for achieving good nutrition and health goals, a device,system, and method for measuring food consumption should differentiatebetween a person's consumption of healthy foods versus unhealthy foods.A device, system, or method can monitor a person's eating habits toencourage consumption of healthy foods and to discourage excessconsumption of unhealthy foods. In an example, one or more of thefollowing types of foods, ingredients, and/or nutrients can beclassified as healthy or unhealthy and tracked by this device, system,and method.

In an example, at least one selected type of food, ingredient, ornutrient can be selected from the group consisting of: a specific typeof carbohydrate, a class of carbohydrates, or all carbohydrates; aspecific type of sugar, a class of sugars, or all sugars; a specifictype of fat, a class of fats, or all fats; a specific type ofcholesterol, a class of cholesterols, or all cholesterols; a specifictype of protein, a class of proteins, or all proteins; a specific typeof fiber, a class of fiber, or all fiber; a specific sodium compound, aclass of sodium compounds, and all sodium compounds; high-carbohydratefood, high-sugar food, high-fat food, fried food, high-cholesterol food,high-protein food, high-fiber food, and high-sodium food.

A device, system, and method for monitoring a person's food consumptionis not a panacea for good nutrition, energy balance, and weightmanagement. However, such a device, system, and method can be a usefulpart of an overall strategy for encouraging good nutrition, energybalance, weight management, and health improvement when a person isengaged and motivated to make good use of it.

INTRODUCTION TO THE FIGURES

FIGS. 1 through 18 show examples of how this invention can be embodied,but they do not limit the full generalizability of the claims.

FIGS. 1 through 4 show an example of a device to monitor a person's foodconsumption comprising a smart watch (with a motion sensor) to detecteating events and a smart spoon (with a built-in chemical compositionsensor), wherein the person is prompted to use the smart spoon to eatfood when the smart watch detects an eating event.

FIGS. 5 through 8 show an example of a device to monitor a person's foodconsumption comprising a smart watch (with a motion sensor) to detecteating events and a smart spoon (with a built-in camera), wherein theperson is prompted to use the smart spoon to take pictures of food whenthe smart watch detects an eating event.

FIGS. 9 through 12 show an example of a device to monitor a person'sfood consumption comprising a smart watch (with a motion sensor) todetect eating events and a smart phone (with a built-in camera), whereinthe person is prompted to use the smart phone to take pictures of foodwhen the smart watch detects an eating event.

FIGS. 13 through 15 show an example of a device to monitor a person'sfood consumption comprising a smart necklace (with a microphone) todetect eating events and a smart phone (with a built-in camera), whereinthe person is prompted to use the smart phone to take pictures of foodwhen the smart necklace detects an eating event.

FIGS. 16 through 18 show an example of a device to monitor a person'sfood consumption comprising a smart necklace (with a microphone) todetect eating events and a smart spoon (with a built-in chemicalcomposition sensor), wherein the person is prompted to use the smartspoon to eat food when the smart necklace detects an eating event.

DETAILED DESCRIPTION OF THE FIGURES

1. Overall Strategy for Good Nutrition and Energy Balance

A device, system, or method for measuring a person's consumption of atleast one selected type of food, ingredient, and/or nutrient is not apanacea for good nutrition, energy balance, and weight management, butit can be a useful part of an overall strategy for encouraging goodnutrition, energy balance, weight management, and health improvement.Although such a device, system, or method is not sufficient to ensureenergy balance and good health, it can be very useful in combinationwith proper exercise and other good health behaviors. Such a device,system, or method can help a person to track and modify their eatinghabits as part of an overall system for good nutrition, energy balance,weight management, and health improvement.

In an example, at least one component of such a device can be worn on aperson's body or clothing. A wearable food-consumption monitoring deviceor system can operate in a more-consistent manner than an entirelyhand-held food-consumption monitoring device, while avoiding thepotential invasiveness and expense of a food-consumption monitoringdevice that is implanted within the body.

Information from a food-consumption monitoring device that measures aperson's consumption of at least one selected type of food, ingredient,and/or nutrient can be combined with information from a separate caloricexpenditure monitoring device that measures a person's caloricexpenditure to comprise an overall system for energy balance, fitness,weight management, and health improvement. In an example, afood-consumption monitoring device can be in wireless communication witha separate fitness monitoring device. In an example, capability formonitoring food consumption can be combined with capability formonitoring caloric expenditure within a single device. In an example, asingle device can be used to measure the types and amounts of food,ingredients, and/or nutrients that a person consumes as well as thetypes and durations of the calorie-expending activities in which theperson engages.

Information from a food-consumption monitoring device that measures aperson's consumption of at least one selected type of food, ingredient,and/or nutrient can also be combined with a computer-to-human interfacethat provides feedback to encourage the person to eat healthy foods andto limit excess consumption of unhealthy foods. In an example, afood-consumption monitoring device can be in wireless communication witha separate feedback device that modifies the person's eating behavior.In an example, capability for monitoring food consumption can becombined with capability for providing behavior-modifying feedbackwithin a single device. In an example, a single device can be used tomeasure the selected types and amounts of foods, ingredients, and/ornutrients that a person consumes and to provide visual, auditory,tactile, or other feedback to encourage the person to eat in a healthiermanner.

A combined device and system for measuring and modifying caloric intakeand caloric expenditure can be a useful part of an overall approach forgood nutrition, energy balance, fitness, weight management, and goodhealth. As part of such an overall system, a device that measures aperson's consumption of at least one selected type of food, ingredient,and/or nutrient can play a key role in helping that person to achievetheir goals with respect to proper nutrition, food consumptionmodification, energy balance, weight management, and good healthoutcomes.

2. Selected Types of Foods, Ingredients, and Nutrients

In order to be really useful for achieving good nutrition and healthgoals, a device and method for measuring a person's consumption of atleast one selected type of food, ingredient, and/or nutrient should beable to differentiate between a person's consumption of healthy foods vsunhealthy foods. This requires the ability to identify consumption ofselected types of foods, ingredients, and/or nutrients, as well asestimating the amounts of such consumption. It also requires selectionof certain types and/or amounts of food, ingredients, and/or nutrientsas healthy vs. unhealthy.

Generally, the technical challenges of identifying consumption ofselected types of foods, ingredients, and/or nutrients are greater thanthe challenges of identifying which types are healthy or unhealthy.Accordingly, while this disclosure covers both food identification andclassification, it focuses in greatest depth on identification ofconsumption of selected types of foods, ingredients, and nutrients. Inthis disclosure, food consumption is broadly defined to includeconsumption of liquid beverages and gelatinous food as well as solidfood.

In an example, a device can identify consumption of at least oneselected type of food. In such an example, selected types of ingredientsor nutrients can be estimated indirectly using a database that linkscommon types and amounts of food with common types and amounts ofingredients or nutrients. In another example, a device can directlyidentify consumption of at least one selected type of ingredient ornutrient. The latter does not rely on estimates from a database, butdoes require more complex ingredient-specific or nutrient-specificsensors. Since the concepts of food identification, ingredientidentification, and nutrient identification are closely related, weconsider them together for many portions of this disclosure, although weconsider them separately in some sections for greater methodologicaldetail. Various embodiments of the device and method disclosed hereincan identify specific nutrients indirectly (through food identificationand use of a database) or directly (through the use of nutrient-specificsensors).

Many people consume highly-processed foods whose primary ingredientsinclude multiple types of sugar. The total amount of sugar is oftenobscured or hidden, even from those who read ingredients on labels.Sometimes sugar is disguised as “evaporated cane syrup.” Sometimesdifferent types of sugar are labeled as different ingredients (such as“plain sugar,” “brown sugar,” “maltose”, “dextrose,” and “evaporatedcane syrup”) in a single food item. In such cases, “sugar” does notappear as the main ingredient. However, when one adds up all thedifferent types of sugar in different priority places on the ingredientlist, then sugar really is the main ingredient. These highly-processedconglomerations of sugar (often including corn syrup, fats, and/orcaffeine) often have colorful labels with cheery terms like “100%natural” or “high-energy.” However, they are unhealthy when eaten in thequantities to which many Americans have become accustomed. It is nowonder that there is an obesity epidemic. The device and methoddisclosed herein is not be fooled by deceptive labeling of ingredients.

In various examples, a device for measuring a person's consumption ofone or more selected types of foods, ingredients, and/or nutrients canmeasure one or more types selected from the group consisting of: aselected type of carbohydrate, a class of carbohydrates, or allcarbohydrates; a selected type of sugar, a class of sugars, or allsugars; a selected type of fat, a class of fats, or all fats; a selectedtype of cholesterol, a class of cholesterols, or all cholesterols; aselected type of protein, a class of proteins, or all proteins; aselected type of fiber, a class of fiber, or all fibers; a specificsodium compound, a class of sodium compounds, or all sodium compounds;high-carbohydrate food, high-sugar food, high-fat food, fried food,high-cholesterol food, high-protein food, high-fiber food, and/orhigh-sodium food.

In various examples, a device for measuring a person's consumption ofone or more selected types of foods, ingredients, and/or nutrients canmeasure one or more types selected from the group consisting of: simplecarbohydrates, simple sugars, saturated fat, trans fat, Low DensityLipoprotein (LDL), and salt. In an example, a device for measuringconsumption of a selected nutrient can measure a person's consumption ofsimple carbohydrates. In an example, a device for measuring consumptionof a selected nutrient can measure a person's consumption of simplesugars. In an example, a device for measuring consumption of a selectednutrient can measure a person's consumption of saturated fats. In anexample, a device for measuring consumption of a selected nutrient canmeasure a person's consumption of trans fats. In an example, a devicefor measuring consumption of a selected nutrient can measure a person'sconsumption of Low Density Lipoprotein (LDL). In an example, a devicefor measuring consumption of a selected nutrient can measure a person'sconsumption of sodium.

In various examples, a food-identifying sensor can detect one or morenutrients selected from the group consisting of: amino acid or protein(a selected type or general class), carbohydrate (a selected type orgeneral class, such as single carbohydrates or complex carbohydrates),cholesterol (a selected type or class, such as HDL or LDL), dairyproducts (a selected type or general class), fat (a selected type orgeneral class, such as unsaturated fat, saturated fat, or trans fat),fiber (a selected type or class, such as insoluble fiber or solublefiber), mineral (a selected type), vitamin (a selected type), nuts (aselected type or general class, such as peanuts), sodium compounds (aselected type or general class), sugar (a selected type or generalclass, such as glucose), and water. In an example, food can beclassified into general categories such as fruits, vegetables, or meat.

In an example, a device for measuring a person's consumption of aselected nutrient can measure a person's consumption of food that ishigh in simple carbohydrates. In an example, a device for measuringconsumption of a selected nutrient can measure a person's consumption offood that is high in simple sugars. In an example, a device formeasuring consumption of a selected nutrient can measure a person'sconsumption of food that is high in saturated fats. In an example, adevice for measuring consumption of a selected nutrient can measure aperson's consumption of food that is high in trans fats. In an example,a device for measuring consumption of a selected nutrient can measure aperson's consumption of food that is high in Low Density Lipoprotein(LDL). In an example, a device for measuring consumption of a selectednutrient can measure a person's consumption of food that is high insodium.

In an example, a device for measuring a person's consumption of aselected nutrient can measure a person's consumption of food wherein ahigh proportion of its calories comes from simple carbohydrates. In anexample, a device for measuring consumption of a selected nutrient canmeasure a person's consumption of food wherein a high proportion of itscalories comes from simple sugars. In an example, a device for measuringconsumption of a selected nutrient can measure a person's consumption offood wherein a high proportion of its calories comes from saturatedfats. In an example, a device for measuring consumption of a selectednutrient can measure a person's consumption of food wherein a highproportion of its calories comes from trans fats. In an example, adevice for measuring consumption of a selected nutrient can measure aperson's consumption of food wherein a high proportion of its caloriescomes from Low Density Lipoprotein (LDL). In an example, a device formeasuring consumption of a selected nutrient can measure a person'sconsumption of food wherein a high proportion of its weight or volume iscomprised of sodium compounds.

In an example, a device for measuring nutrient consumption can track thequantities of selected chemicals that a person consumes via foodconsumption. In various examples, these consumed chemicals can beselected from the group consisting of carbon, hydrogen, nitrogen,oxygen, phosphorus, and sulfur. In an example, a food-identifying devicecan selectively detect consumption of one or more types of unhealthyfood, wherein unhealthy food is selected from the group consisting of:food that is high in simple carbohydrates; food that is high in simplesugars; food that is high in saturated or trans fat; fried food; foodthat is high in Low Density Lipoprotein (LDL); and food that is high insodium.

In a broad range of examples, a food-identifying sensor can measure oneor more types selected from the group consisting of: a selected food,ingredient, or nutrient that has been designated as unhealthy by ahealth care professional organization or by a specific health careprovider for a specific person; a selected substance that has beenidentified as an allergen for a specific person; peanuts, shellfish, ordairy products; a selected substance that has been identified as beingaddictive for a specific person; alcohol; a vitamin or mineral; vitaminA, vitamin B1, thiamin, vitamin B12, cyanocobalamin, vitamin B2,riboflavin, vitamin C, ascorbic acid, vitamin D, vitamin E, calcium,copper, iodine, iron, magnesium, manganese, niacin, pantothenic acid,phosphorus, potassium, riboflavin, thiamin, and zinc; a selected type ofcarbohydrate, class of carbohydrates, or all carbohydrates; a selectedtype of sugar, class of sugars, or all sugars; simple carbohydrates,complex carbohydrates; simple sugars, complex sugars, monosaccharides,glucose, fructose, oligosaccharides, polysaccharides, starch, glycogen,disaccharides, sucrose, lactose, starch, sugar, dextrose, disaccharide,fructose, galactose, glucose, lactose, maltose, monosaccharide,processed sugars, raw sugars, and sucrose; a selected type of fat, classof fats, or all fats; fatty acids, monounsaturated fat, polyunsaturatedfat, saturated fat, trans fat, and unsaturated fat; a selected type ofcholesterol, a class of cholesterols, or all cholesterols; Low DensityLipoprotein (LDL), High Density Lipoprotein (HDL), Very Low DensityLipoprotein (VLDL), and triglycerides; a selected type of protein, aclass of proteins, or all proteins; dairy protein, egg protein, fishprotein, fruit protein, grain protein, legume protein, lipoprotein, meatprotein, nut protein, poultry protein, tofu protein, vegetable protein,complete protein, incomplete protein, or other amino acids; a selectedtype of fiber, a class of fiber, or all fiber; dietary fiber, insolublefiber, soluble fiber, and cellulose; a specific sodium compound, a classof sodium compounds, and all sodium compounds; salt; a selected type ofmeat, a class of meats, and all meats; a selected type of vegetable, aclass of vegetables, and all vegetables; a selected type of fruit, aclass of fruits, and all fruits; a selected type of grain, a class ofgrains, and all grains; high-carbohydrate food, high-sugar food,high-fat food, fried food, high-cholesterol food, high-protein food,high-fiber food, and high-sodium food.

In an example, a device for measuring a person's consumption of at leastone specific food, ingredient, and/or nutrient that can analyze foodcomposition can also identify one or more potential food allergens,toxins, or other substances selected from the group consisting of:ground nuts, tree nuts, dairy products, shell fish, eggs, gluten,pesticides, animal hormones, and antibiotics. In an example, a devicecan analyze food composition to identify one or more types of food whoseconsumption is prohibited or discouraged for religious, moral, and/orcultural reasons, such as pork or meat products of any kind.

3. Metrics for Measuring Foods, Ingredients, and Nutrients

Having discussed different ways to classify types of foods, ingredients,and nutrients, we now turn to different metrics for measuring theamounts of foods, ingredients, and nutrients consumed. Overall, amountsor quantities of food, ingredients, and nutrients consumed can bemeasured in terms of volume, mass, or weight. Volume measures how muchspace the food occupies. Mass measures how much matter the foodcontains. Weight measures the pull of gravity on the food. The conceptsof mass and weight are related, but not identical. Food, ingredient, ornutrient density can also be measured, sometimes as a step towardmeasuring food mass.

Volume can be expressed in metric units (such as cubic millimeters,cubic centimeters, or liters) or U.S. (historically English) units (suchas cubic inches, teaspoons, tablespoons, cups, pints, quarts, gallons,or fluid ounces). Mass (and often weight in colloquial use) can beexpressed in metric units (such as milligrams, grams, and kilograms) orU.S. (historically English) units (ounces or pounds). The density ofspecific ingredients or nutrients within food is sometimes measured interms of the volume of specific ingredients or nutrients per total foodvolume or measured in terms of the mass of specific ingredients ornutrients per total food mass.

In an example, the amount of a specific ingredient or nutrient within (aportion of) food can be measured directly by a sensing mechanism. In anexample, the amount of a specific ingredient or nutrient within (aportion of) food can be estimated indirectly by measuring the amount offood and then linking this amount of food to amounts of ingredients ornutrients using a database that links specific foods with standardamounts of ingredients or nutrients.

In an example, an amount of a selected type of food, ingredient, ornutrient consumed can be expressed as an absolute amount. In an example,an amount of a selected type of food, ingredient, or nutrient consumedcan be expressed as a percentage of a standard amount. In an example, anamount of a selected type of food, ingredient, or nutrient consumed canbe displayed as a portion of a standard amount such as in a bar chart,pie chart, thermometer graphic, or battery graphic.

In an example, a standard amount can be selected from the groupconsisting of: daily recommended minimum amount; daily recommendedmaximum amount or allowance; weekly recommended minimum amount; weeklyrecommended maximum amount or allowance; target amount to achieve ahealth goal; and maximum amount or allowance per meal. In an example, astandard amount can be a Reference Daily Intake (RDI) value or a DailyReference Value.

In an example, the volume of food consumed can be estimated by analyzingone or more pictures of that food. In an example, volume estimation caninclude the use of a physical or virtual fiduciary marker or object ofknown size for estimating the size of a portion of food. In an example,a physical fiduciary marker can be placed in the field of view of animaging system for use as a point of reference or a measure. In anexample, this fiduciary marker can be a plate, utensil, or otherphysical place setting member of known size. In an example, thisfiduciary marker can be created virtually by the projection of coherentlight beams. In an example, a device can project (laser) light pointsonto food and, in conjunction with infrared reflection or focaladjustment, use those points to create a virtual fiduciary marker. Afiduciary marker may be used in conjunction with a distance-findingmechanism (such as infrared range finder) that determines the distancefrom the camera and the food.

In an example, volume estimation can include obtaining video images offood or multiple still pictures of food in order to obtain pictures offood from multiple perspectives. In an example, pictures of food frommultiple perspectives can be used to create three-dimensional orvolumetric models of that food in order to estimate food volume. In anexample, such methods can be used prior to food consumption and againafter food consumption, in order to estimate the volume of food consumedbased on differences in food volume measured. In an example, food volumeestimation can be done by analyzing one or more pictures of food before(and after) consumption. In an example, multiple pictures of food fromdifferent angles can enable three-dimensional modeling of food volume.In an example, multiple pictures of food at different times (such asbefore and after consumption) can enable estimation of the amount ofproximal food that is actually consumed vs. just being served inproximity to the person.

In a non-imaging example of food volume estimation, a utensil or otherapportioning device can be used to divide food into mouthfuls. Then, thenumber of times that the utensil is used to bring food up to theperson's mouth can be tracked. Then, the number of utensil motions ismultiplied times the estimated volume of food per mouthful in order toestimate the cumulative volume of food consumed. In an example, thenumber of hand motions or mouth motions can be used to estimate thequantity of food consumed. In an example, a motion sensor worn on aperson's wrist or incorporated into a utensil can measure the number ofhand-to-mouth motions. In an example, a motion sensor, sound sensor, orelectromagnetic sensor in communication with a person's mouth canmeasure the number of chewing motions which, in turn, can be used toestimate food volume.

In an example, a device for measuring a person's consumption of one ormore selected types of foods, ingredients, or nutrients can measure theweight or mass of food that the person consumes. In an example, a deviceand method for measuring consumption of one or more selected types offoods, ingredients, or nutrients can include a food scale that measuresthe weight of food. In an example a food scale can measure the weight offood prior to consumption and the weight of unconsumed food remainingafter consumption in order to estimate the weight of food consumed basedon the difference in pre vs. post consumption measurements. In anexample, a food scale can be a stand-alone device. In an example, a foodscale can be incorporated into a plate, glass, cup, glass coaster, placemat, or other place setting. In an example a plate can include differentsections which separately measure the weights of different foods on theplate. In an example, a food scale embedded into a place setting orsmart utensil can automatically transmit data concerning food weight toa computer.

In an example, a food scale can be incorporated into a smart utensil. Inan example, a food scale can be incorporated into a utensil rest onwhich a utensil is placed for each bite or mouthful. In an example, afood scale can be incorporated into a smart utensil which tracks thecumulative weight of cumulative mouthfuls of food during an eatingevent. In an example, a smart utensil can approximate the weight ofmouthfuls of food by measuring the effect of food carried by the utensilon an accelerometer or other inertial sensor. In an example, a smartutensil can incorporate a spring between the food-carrying portion andthe hand-held portion of a utensil and food weight can be estimated bymeasuring distension of the spring as food is brought up to a person'smouth.

In an example, a smart utensil can use an inertial sensor,accelerometer, or strain gauge to estimate the weight of thefood-carrying end of utensil at a first time (during an upswing motionas the utensil carries a mouthful of food up to the person's mouth), canuse this sensor to estimate the weight of the food-carrying end of theutensil at a second time (during a downswing motion as the person lowersthe utensil from their mouth), and can estimate the weight of themouthful of food by calculating the difference in weight between thefirst and second times.

In an example, a device or system can measure nutrient density orconcentration as part of an automatic food, ingredient, or nutrientidentification method. In an example, such nutrient density can beexpressed as the average amount of a specific ingredient or nutrient perunit of food weight. In an example, such nutrient density can beexpressed as the average amount of a specific ingredient or nutrient perunit of food volume. In an example, food density can be estimated byinteracting food with light, sound, or electromagnetic energy andmeasuring the results of this interaction. Such interaction can includeenergy absorption or reflection.

In an example, nutrient density can be determined by reading a label onpackaging associated with food consumed. In an example, nutrient densitycan be determined by receipt of wirelessly transmitted information froma grocery store display, electronically-functional restaurant menu, orvending machine. In an example, food density can be estimated byultrasonic scanning of food. In an example, food density and food volumecan be jointly analyzed to estimate food weight or mass.

In an example, for some foods with standardized sizes (such as foodsthat are manufactured in standard sizes at high volume), food weight canbe estimated as part of food identification. In an example, informationconcerning the weight of food consumed can be linked to nutrientquantities in a computer database in order to estimate cumulativeconsumption of selected types of nutrients.

In an example, a method for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient can comprisemonitoring changes in the volume or weight of food at a reachablelocation near the person. In an example, pictures of food can be takenat multiple times before, during, and after food consumption in order tobetter estimate the amount of food that the person actually consumes,which can differ from the amount of food served to the person or theamount of food left over after the person eats. In an example, estimatesof the amount of food that the person actually consumes can be made bydigital image subtraction and/or 3D modeling. In an example, changes inthe volume or weight of nearby food can be correlated with hand motionsin order to estimate the amount of food that a person actually eats. Inan example, a device can track the cumulative number of hand-to-mouthmotions, number of chewing motions, or number of swallowing motions. Inan example, estimation of food consumed can also involve asking theperson whether they ate all the food that was served to them.

In an example, a device and method for measuring a person's consumptionof at least one selected type of food, ingredient, or nutrient cancollect data that enables tracking the cumulative amount of a type offood, ingredient, or nutrient which the person consumes during a periodof time (such as an hour, day, week, or month) or during a particulareating event. In an example, the time boundaries of a particular eatingevent can be defined by a maximum time between chews or mouthfuls duringa meal and/or a minimum time between chews or mouthfuls between meals.In an example, the time boundaries of a particular eating event can bedefined by Fourier Transformation analysis of the variable frequenciesof chewing, swallowing, or biting during meals vs. between meals.

In an example, a device and method for measuring a person's consumptionof at least one selected type of food, ingredient, or nutrient can trackthe cumulative amount of that food, ingredient, or nutrient consumed bythe person and provide feedback to the person based on the person'scumulative consumption relative to a target amount. In an example, adevice can provide negative feedback when a person exceeds a targetamount of cumulative consumption. In an example, a device and system cansound an alarm or provide other real-time feedback to a person when thecumulative consumed amount of a selected type of food, ingredient, ornutrient exceeds an allowable amount (in total, per meal, or per unit oftime).

In various examples, a target amount of consumption can be based on oneor more factors selected from the group consisting of: the selected typeof selected food, ingredient, or nutrient; amount of this typerecommended by a health care professional or governmental agency;specificity or breadth of the selected nutrient type; the person's age,gender, and/or weight; the person's diagnosed health conditions; theperson's exercise patterns and/or caloric expenditure; the person'sphysical location; the person's health goals and progress thus fartoward achieving them; one or more general health status indicators;magnitude and/or certainty of the effects of past consumption of theselected nutrient on the person's health; the amount and/or duration ofthe person's consumption of healthy food or nutrients; changes in theperson's weight; time of day; day of the week; occurrence of a holidayor other occasion involving special meals; dietary plan created for theperson by a health care provider; input from a social network and/orbehavioral support group; input from a virtual health coach; healthinsurance copay and/or health insurance premium; financial payments,constraints, and/or incentives; cost of food; speed or pace of nutrientconsumption; and accuracy of the sensor in detecting the selectednutrient.

4. Food Consumption and Nutrient Identification Sensors

A device and method for measuring a person's consumption of at least oneselected type of food, ingredient, or nutrient can include: a generalfood-consumption monitor that detects when a person is probably eating,but does not identify the selected types of foods, ingredients, ornutrients that the person is eating; and a food-identifying sensor thatidentifies the person's consumption of at least one selected type offood, ingredient, or nutrient.

In an example, operation of a food-identifying sensor can be triggeredby the results of a general food-consumption monitor. In an example, ageneral food-consumption monitor with low privacy intrusion (but lowfood identification accuracy) can operate continually and trigger theoperation of a food-identifying sensor with high privacy intrusion (buthigh food identification accuracy) when the person is eating. In anexample, a general food-consumption monitor with low privacy intrusion(but low power or resource requirements) can operate continually andtrigger the operation of a food-identifying sensor with high privacyintrusion (but high power or resource requirements) when the person iseating. In an example, the combination of a general food-consumptionmonitor and a food-identifying sensor can achieve relatively-high foodidentification accuracy with relatively-low privacy intrusion orresource requirements.

In an example, a food-consumption monitor or food-identifying sensor canmeasure food weight, mass, volume, or density. In an example, such asensor can be a scale, strain gauge, or inertial sensor. In an example,such a sensor can measure the weight or mass of an entire meal, aportion of one type of food within that meal, or a mouthful of a type offood that is being conveyed to a person's mouth. In general, a weight,mass, or volume sensor is more useful for general detection of foodconsumption and food amount than it is for identification of consumptionof selected types of foods, ingredients, and nutrients. However, it canbe very useful when used in combination with a specific food-identifyingsensor.

In an example, a food-consumption monitor can be a motion sensor. Invarious examples, a motion sensor can be selected from the groupconsisting of: bubble accelerometer, dual-axial accelerometer,electrogoniometer, gyroscope, inclinometer, inertial sensor, multi-axisaccelerometer, piezoelectric sensor, piezo-mechanical sensor, pressuresensor, proximity detector, single-axis accelerometer, strain gauge,stretch sensor, and tri-axial accelerometer. In an example, a motionsensor can collect data concerning the movement of a person's wrist,hand, fingers, arm, head, mouth, jaw, or neck. In an example, analysisof this motion data can be used to identify when the person is probablyeating. In general, a motion sensor is more useful for general detectionof food consumption and food amount than it is for identification ofconsumption of selected types of foods, ingredients, and nutrients.However, it can be very useful when used in combination with a specificfood-identifying sensor.

In an example, there can be an identifiable pattern of movement that ishighly-associated with food consumption and a motion sensor can monitora person's movements to identify times when the person is probablyeating. In an example, this movement can include repeated movement of aperson's hand up to their mouth. In an example, this movement caninclude a combination of three-dimensional roll, pitch, and yaw by aperson's wrist and/or hand. In an example, this movement can includerepeated bending of a person's elbow. In an example, this movement caninclude repeated movement of a person's jaws. In an example, thismovement can include peristaltic motion of the person's esophagus thatis detectable from contact with a person's neck.

In an example, a motion sensor can be used to estimate the quantity offood consumed based on the number of motion cycles. In an example, amotion sensor can be used to estimate the speed of food consumptionbased on the speed or frequency of motion cycles. In an example, aproximity sensor can detect when a person's hand gets close to theirmouth. In an example, a proximity sensor can detect when a wrist (orhand or finger) is in proximity to a person's mouth. However, aproximity detector can be less useful than a motion detector because itdoes not identify complex three-dimensional motions that candifferentiate eating from other hand-to-mouth motions such as coughing,yawning, smoking, and tooth brushing.

In various examples, a device to measure a person's consumption of atleast one selected type of food, ingredient, or nutrient can include amotion sensor that collects data concerning movement of the person'sbody. In an example, this data can be used to detect when a person isconsuming food. In an example, this data can be used to aid in theidentification of what types and amounts of food the person isconsuming. In an example, analysis of this data can be used to triggeradditional data collection to resolve uncertainty concerning the typesand amounts of food that the person is consuming.

In an example, a motion sensor can include one or more accelerometers,inclinometers, electrogoniometers, and/or strain gauges. In an example,movement of a person's body that can be monitored and analyzed can beselected from the group consisting of: finger movements, hand movements,wrist movements, arm movements, elbow movements, eye movements, and headmovements; tilting movements, lifting movements; hand-to-mouthmovements; angles of rotation in three dimensions around the center ofmass known as roll, pitch and yaw; and Fourier Transformation analysisof repeated body member movements. In an example, each hand-to-mouthmovement that matches a certain pattern can be used to estimate bite ormouthful of food. In an example, the speed of hand-to-mouth movementsthat match a certain pattern can be used to estimate eating speed. In anexample, this pattern can include upward and tilting hand movement,followed by a pause, following by a downward and tilting hand movement.

In an example, a motion sensor that is used to detect food consumptioncan be worn on a person's wrist, hand, arm, or finger. In an example, amotion sensor can be incorporated into a smart watch, fitness watch, orwatch phone. In an example, a fitness watch that already uses anaccelerometer to measure motion for estimating caloric expenditure canalso use an accelerometer to detect (and estimate the quantity of) foodconsumption.

Motion-sensing devices that are worn on a person's wrist, hand, arm, orfinger can continuously monitor a person's movements to detect foodconsumption with high compliance and minimal privacy intrusion. They donot require that a person carry a particular piece of electronicequipment everywhere they go and consistently bring that piece ofelectronic equipment out for activation each time that they eat a mealor snack. However, a motion-detecting device that is worn constantly ona person's wrist, hand, arm, or finger can be subject to false alarmsdue to motions (such as coughing, yawning, smoking, and tooth brushing)that can be similar to eating motions. To the extent that there is adistinctive pattern of hand and/or arm movement associated with bringingfood up to one's mouth, such a device can detect when food consumptionis occurring.

In an example, a motion-sensing device that is worn on a person's wrist,hand, arm, or finger can measure how rapidly or often the person bringstheir hand up to their mouth. A common use of such information is toencourage a person to eat at a slower pace. The idea that a person willeat less if they eat at a slower pace is based on the lag between foodconsumption and the feeling of satiety from internal gastric organs. Ifa person eats slower, then they will tend to not overeat past the pointof internal identification of satiety.

In an example, a smart watch, fitness watch, watch phone, smart ring, orsmart bracelet can measure the speed, pace, or rate at which a personbrings food up to their mouth while eating and provide feedback to theperson to encourage them to eat slower if the speed, pace, or rate ishigh. In an example, feedback can be sound-based, such as an alarm,buzzer, or computer-generated voice. In an example, feedback can betactile, such as vibration or pressure. In an example, such feedback canbe visual, such as a light, image, or display screen. In an alternativeexample, eating speed can be inferred indirectly by a plate, dish, bowl,glass or other place setting member that measures changes in the weightof food on the member. Negative feedback can be provided to the personif the weight of food on the plate, dish, bowl, or glass decreases in amanner that indicates that food consumption is too fast.

In an example, a motion sensor that is used to detect food consumptioncan be incorporated into, or attached to, a food utensil such as a forkor spoon. A food utensil with a motion sensor can be less prone to falsealarms than a motion sensor worn on a person's wrist, hand, arm, orfinger because the utensil is only used when the person eats food. Sincethe utensil is only used for food consumption, analysis of complexmotion and differentiation of food consumption actions vs. other handgestures is less important with a utensil than it is with a device thatis worn on the person's body. In an example, a motion sensor can beincorporated into a smart utensil. In an example, a smart utensil canestimate the amount of food consumed by the number of hand-to-mouthmotions (combined with information concerning how much food is conveyedby the utensil with each movement). In an example, a smart utensil canencourage a person to eat slower. The idea is that if the person eatsmore slowly, then they will tend to not overeat past the point ofinternal identification of satiety.

In an example, a food-consumption monitor or food-identifying sensor canbe a light-based sensor that records the interaction between light andfood. In an example, a light-based sensor can be a camera, mobile phone,or other conventional imaging device that takes plain-light pictures offood. In an example, a light-based food consumption or identificationsensor can comprise a camera that takes video pictures or still picturesof food. In an example, such a camera can take pictures of theinteraction between a person and food, including food apportionment,hand-to-mouth movements, and chewing movements.

In an example, a device and method for measuring a person's consumptionof at least one selected type of food, ingredient, or nutrient caninclude a camera, or other picture-taking device, that takes pictures offood. In the following section, we discuss different examples of how acamera or other imaging-device can be used to take pictures of food andhow those pictures can be analyzed to identify the types and amounts offood consumed. After that section, we discuss some other light-basedapproaches to food identification (such as spectroscopy) that do notrely on conventional imaging devices and plain-light food pictures.

A food-consumption monitor or food-identifying sensor can be a camera orother imaging device that is carried and held by a person. In anexample, a camera that is used for food identification can be part of amobile phone, cell phone, electronic tablet, or smart food utensil. Inan example, a food-consumption monitor or food-identifying sensor can bea camera or other imaging device that is worn on a person's body orclothing. In an example, a camera can be incorporated into a smartwatch, smart bracelet, smart button, or smart necklace.

In an example, a camera that is used for monitoring food consumptionand/or identifying consumption of at least one selected type of food,ingredient, or nutrient can be a dedicated device that is specificallydesigned for this purpose. In an example, a camera that is used formonitoring food consumption and/or identifying consumption of specificfoods can be a part of a general purpose device (such as a mobile phone,cell phone, electronic tablet, or digital camera) and in wirelesscommunication with a dedicated device for monitoring food consumptionand identifying specific food types.

In an example, use of a hand-held camera, mobile phone, or other imagingdevice to identify food depends on a person's manually aiming andtriggering the device for each eating event. In an example, the personmust bring the imaging device with them to each meal or snack, orient ittoward the food to be consumed, and activate taking a picture of thefood by touch or voice command. In an example, a camera, smart watch,smart necklace or other imaging device that is worn on a person's bodyor clothing can move passively as the person moves. In an example, thefield of vision of an imaging device that is worn on a person's wrist,hand, arm, or finger can move as the person brings food up to theirmouth when eating. In an example, such an imaging device can passivelycapture images of a reachable food source and interaction between foodand a person's mouth.

In another example, the imaging vector and/or focal range of an imagingdevice worn on a person's body or clothing can be actively anddeliberately adjusted to better track the person's hands and mouth tobetter monitor for possible food consumption. In an example, a devicecan optically scan the space surrounding the person for reachable foodsources, hand-to-food interaction, and food-to-mouth interaction. In anexample, in the interest of privacy, an imaging device that is worn on aperson's body or clothing can only take pictures when some other sensoror information indicates that the person is probably eating.

In an example, a camera that is used for identifying food consumptioncan have a variable focal length. In an example, the imaging vectorand/or focal distance of a camera can be actively and automaticallyadjusted to focus on: the person's hands, space surrounding the person'shands, a reachable food source, a food package, a menu, the person'smouth, and the person's face. In an example, in the interest of privacy,the focal length of a camera can be automatically adjusted in order tofocus on food and not other people.

In an example, a device for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient can include animaging component that the person must manually aim toward food andmanually activate for taking food pictures (such as through touch orvoice commands). In an example, the taking of food pictures in thismanner requires at least one specific voluntary human action associatedwith each food consumption event, apart from the actual act of eating,in order to take pictures of food during that food consumption event. Inan example, such specific voluntary human actions can be selected fromthe group consisting of: transporting a mobile imaging device to a meal;aiming an imaging device at food; clicking a button to activate picturetaking; touching a screen to activate picture taking; and speaking avoice command to activate picture taking.

In an example, a device for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient can prompt a personto take pictures of food when a non-imaging sensor or other source ofinformation indicates that the person is probably eating. In analternative example, a device for measuring a person's consumption of atleast one selected type of food, ingredient, or nutrient canautomatically take pictures of food consumed without the need forspecific action by the person in association with a specific eatingevent apart from the act of eating.

In an example, a device and method for measuring food consumption caninclude taking multiple pictures of food. In an example, such a deviceand method can include taking pictures of food from at least twodifferent angles in order to better segment a meal into different typesof foods, estimate the three-dimensional volume of each type of food,and control for lighting and shading differences. In an example, acamera or other imaging device can take pictures of food from multipleperspectives to create a virtual three-dimensional model of food inorder to determine food volume. In an example, an imaging device canestimate the quantities of specific foods from pictures or images ofthose foods by volumetric analysis of food from multiple perspectivesand/or by three-dimensional modeling of food from multiple perspectives.

In an example, a camera can use an object of known size within its fieldof view as a fiduciary marker in order to measure the size or scale offood. In an example, a camera can use projected laser beams to create avirtual or optical fiduciary marker in order to measure food size orscale. In an example, pictures of food can be taken at different times.In an example, a camera can be used to take pictures of food before andafter consumption. The amount of food that a person actually consumes(not just the amount ordered or served) can be estimated by thedifference in observed food volume from the pictures before and afterconsumption.

In an example, images of food can be automatically analyzed in order toidentify the types and quantities of food consumed. In an example,pictures of food taken by a camera or other picture-taking device can beautomatically analyzed to estimate the types and amounts of specificfoods, ingredients, or nutrients that a person is consumes. In anexample, an initial stage of an image analysis system can compriseadjusting, normalizing, or standardizing image elements for better foodsegmentation, identification, and volume estimation. These elements caninclude: color, texture, shape, size, context, geographic location,adjacent food, place setting context, and temperature (infrared). In anexample, a device can identify specific foods from pictures or images byimage segmentation, color analysis, texture analysis, and patternrecognition.

In various examples, automatic identification of food types andquantities can be based on: color and texture analysis; imagesegmentation; image pattern recognition; volumetric analysis based on afiduciary marker or other object of known size; and/or three-dimensionalmodeling based on pictures from multiple perspectives. In an example, adevice can collect food images that are used to extract a vector of foodparameters (such as color, texture, shape, and size) that areautomatically associated with vectors of food parameters in a databaseof such parameters for food identification.

In an example, a device can collect food images that are automaticallyassociated with images of food in a food image database for foodidentification. In an example, specific ingredients or nutrients thatare associated with these selected types of food can be estimated basedon a database linking foods to ingredients and nutrients. In anotherexample, specific ingredients or nutrients can be measured directly. Invarious examples, a device for measuring consumption of food,ingredient, or nutrients can directly (or indirectly) measureconsumption at least one selected type of food, ingredient, or nutrient.

In an example, food image information can be transmitted from a wearableor hand-held device to a remote location where automatic foodidentification occurs and the results can be transmitted back to thewearable or hand-held device. In an example, identification of the typesand quantities of foods, ingredients, or nutrients that a personconsumes from pictures of food can be a combination of, or interactionbetween, automated identification food methods and human-based foodidentification methods.

We now transition to discussion of light-based methods for measuringfood consumption that do not rely of conventional imaging devices andplain-light images. Probably the simplest such method involvesidentifying food by scanning a barcode or other machine-readable code onthe food's packaging (such as a Universal Product Code or EuropeanArticle Number), on a menu, on a store display sign, or otherwise inproximity to food at the point of food selection, sale, or consumption.In an example, the type of food (and/or specific ingredients ornutrients within the food) can be identified by machine-recognition of afood label, nutritional label, or logo on food packaging, menu, ordisplay sign. However, there are many types of food and food consumptionsituations in which food is not accompanied by such identifyingpackaging. Accordingly, a robust imaged-based device and method formeasuring food consumption should not rely on bar codes or otheridentifying material on food packaging.

In an example, selected types of foods, ingredients, and/or nutrientscan be identified by the patterns of light that are reflected from, orabsorbed by, the food at different wavelengths. In an example, alight-based sensor can detect food consumption or can identifyconsumption of a specific food, ingredient, or nutrient based on thereflection of light from food or the absorption of light by food atdifferent wavelengths. In an example, an optical sensor can detectfluorescence. In an example, an optical sensor can detect whether foodreflects light at a different wavelength than the wavelength of lightshone on food. In an example, an optical sensor can be a fluorescencepolarization immunoassay sensor, chemiluminescence sensor,thermoluminescence sensor, or piezoluminescence sensor.

In an example, a light-based food-identifying sensor can collectinformation concerning the wavelength spectra of light reflected from,or absorbed by, food. In an example, an optical sensor can be achromatographic sensor, spectrographic sensor, analyticalchromatographic sensor, liquid chromatographic sensor, gaschromatographic sensor, optoelectronic sensor, photochemical sensor, andphotocell. In an example, an optical sensor can analyze modulation oflight wave parameters by the interaction of that light with a portion offood. In an example, an optical sensor can detect modulation of lightreflected from, or absorbed by, a receptor when the receptor is exposedto food. In an example, an optical sensor can emit and/or detect whitelight, infrared light, or ultraviolet light.

In an example, a light-based food-identifying sensor can identifyconsumption of a selected type of food, ingredient, or nutrient with aspectral analysis sensor. In various examples, a food-identifying sensorcan identify a selected type of food, ingredient, or nutrient with asensor that detects light reflection spectra, light absorption spectra,or light emission spectra. In an example, a spectral measurement sensorcan be a spectroscopy sensor or a spectrometry sensor. In an example, aspectral measurement sensor can be a white light spectroscopy sensor, aninfrared spectroscopy sensor, a near-infrared spectroscopy sensor, anultraviolet spectroscopy sensor, an ion mobility spectroscopic sensor, amass spectrometry sensor, a backscattering spectrometry sensor, or aspectrophotometer. In an example, light at different wavelengths can beabsorbed by, or reflected off, food and the results can be analyzed inspectral analysis.

In an example, a food-consumption monitor or food-identifying sensor canbe a microphone or other type of sound sensor. In an example, a sensorto detect food consumption and/or identify consumption of a selectedtype of food, ingredient, or nutrient can be a sound sensor. In anexample, a sound sensor can be an air conduction microphone or boneconduction microphone. In an example, a microphone or other sound sensorcan monitor for sounds associated with chewing or swallowing food. In anexample, data collected by a sound sensor can be analyzed todifferentiate sounds from chewing or swallowing food from other types ofsounds such as speaking, singing, coughing, and sneezing.

In an example, a sound sensor can include speech recognition or voicerecognition to receive verbal input from a person concerning food thatthe person consumes. In an example, a sound sensor can include speechrecognition or voice recognition to extract food selecting, ordering,purchasing, or consumption information from other sounds in theenvironment.

In an example, a sound sensor can be worn or held by a person. In anexample, a sound sensor can be part of a general purpose device, such asa cell phone or mobile phone, which has multiple applications. In anexample, a sound sensor can measure the interaction of sound waves (suchas ultrasonic sound waves) and food in order to identify the type andquantity of food that a person is eating.

In an example, a food-consumption monitor or food-identifying sensor canbe a chemical sensor. In an example, a chemical sensor can include areceptor to which at least one specific nutrient-related analyte bindsand this binding action creates a detectable signal. In an example, achemical sensor can include measurement of changes in energy waveparameters that are caused by the interaction of that energy with food.In an example, a chemical sensor can be incorporated into a smartutensil to identify selected types of foods, ingredients, or nutrients.In an example, a chemical sensor can be incorporated into a portablefood probe to identify selected types of foods, ingredients, ornutrients. In an example, a sensor can analyze the chemical compositionof a person's saliva. In an example, a chemical sensor can beincorporated into an intraoral device that analyzes micro-samples of aperson's saliva. In an example, such an intraoral device can be adheredto a person's upper palate.

In various examples, a food-consumption monitor or food-identifyingsensor can be selected from the group consisting of: receptor-basedsensor, enzyme-based sensor, reagent based sensor, antibody-basedreceptor, biochemical sensor, membrane sensor, pH level sensor,osmolality sensor, nucleic acid-based sensor, or DNA/RNA-based sensor; abiomimetic sensor (such as an artificial taste bud or an artificialolfactory sensor), a chemiresistor, a chemoreceptor sensor, aelectrochemical sensor, an electroosmotic sensor, an electrophoresissensor, or an electroporation sensor; a specific nutrient sensor (suchas a glucose sensor, a cholesterol sensor, a fat sensor, a protein-basedsensor, or an amino acid sensor); a color sensor, a colorimetric sensor,a photochemical sensor, a chemiluminescence sensor, a fluorescencesensor, a chromatography sensor (such as an analytical chromatographysensor, a liquid chromatography sensor, or a gas chromatography sensor),a spectrometry sensor (such as a mass spectrometry sensor), aspectrophotometer sensor, a spectral analysis sensor, or a spectroscopysensor (such as a near-infrared spectroscopy sensor); and alaboratory-on-a-chip or microcantilever sensor.

In an example, a food-consumption monitor or food-identifying sensor canbe an electromagnetic sensor. In an example, a device for measuring foodconsumption or identifying specific nutrients can emit and measureelectromagnetic energy. In an example, a device can expose food toelectromagnetic energy and collect data concerning the effects of thisinteraction which are used for food identification. In various examples,the results of this interaction can include measuring absorption orreflection of electromagnetic energy by food. In an example, anelectromagnetic sensor can detect the modulation of electromagneticenergy that is interacted with food.

In an example, an electromagnetic sensor that detects food or nutrientconsumption can detect electromagnetic signals from the body in responseto the consumption or digestion of food. In an example, analysis of thiselectromagnetic energy can help to identify the types of food that aperson consumes. In an example, a device can measure electromagneticsignals emitted by a person's stomach, esophagus, mouth, tongue,afferent nervous system, or brain in response to general foodconsumption. In an example, a device can measure electromagnetic signalsemitted by a person's stomach, esophagus, mouth, tongue, afferentnervous system, or brain in response to consumption of selected types offoods, ingredients, or nutrients.

In various examples, a sensor to detect food consumption or identifyconsumption of a selected type of nutrient can be selected from thegroup consisting of: neuroelectrical sensor, action potential sensor,ECG sensor, EKG sensor, EEG sensor, EGG sensor, capacitance sensor,conductivity sensor, impedance sensor, galvanic skin response sensor,variable impedance sensor, variable resistance sensor, interferometer,magnetometer, RF sensor, electrophoretic sensor, optoelectronic sensor,piezoelectric sensor, and piezocapacitive sensor.

In an example, a sensor to monitor, detect, or sense food consumption orto identify a selected type of food, ingredient, or nutrient consumedcan be pressure sensor or touch sensor. In an example, a pressure ortouch sensor can sense pressure or tactile information from contact withfood that will be consumed. In an example, a pressure or touch sensorcan be incorporated into a smart food utensil or food probe. In anexample, a pressure or touch based sensor can be incorporated into a padon which a food utensil is placed between mouthfuls or when not in use.In an example, a pressure or touch sensor can sense pressure or tactileinformation from contact with a body member whose internal pressure orexternal shape is affected by food consumption. In various examples, apressure or touch sensor can be selected from the group consisting of:food viscosity sensor, blood pressure monitor, muscle pressure sensor,button or switch on a food utensil, jaw motion pressure sensor, andhand-to-mouth contact sensor.

In an example, a food-consumption monitor or food-identifying sensor canbe a thermal energy sensor. In an example, a thermal sensor can detector measure the temperature of food. In an example, a thermal sensor candetect or measure the temperature of a portion of a person's bodywherein food consumption changes the temperature of this member. Invarious examples, a food-consumption monitor can be selected from thegroup consisting of: a thermometer, a thermistor, a thermocouple, and aninfrared energy detector.

In an example, a food-consumption monitor or food-identifying sensor canbe a location sensor. In an example, such a sensor can be geographiclocation sensor or an intra-building location sensor. A device fordetecting food consumption and/or indentifying a selected type of food,ingredient, or nutrient consumed can use information concerning aperson's location as part of the means for food consumption detectionand/or food identification. In an example, a device can identify when aperson in a geographic location that is associated with probable foodconsumption. In an example, a device can use information concerning theperson's geographic location as measured by a global positioning systemor other geographic location identification system. In an example, if aperson is located at a restaurant with a known menu or at a store with aknown food inventory, then information concerning this menu or foodinventory can be used to narrow down the likely types of food beingconsumed. In an example, if a person is located at a restaurant, thenthe sensitivity of automated detection of food consumption can beadjusted. In an example, if a person is located at a restaurant orgrocery store, then visual, auditory, or other information collected bya sensor can be interpreted within the context of that location.

In an example, a device can identify when a person is in a locationwithin a building that is associated with probable food consumption. Inan example, if a person is in a kitchen or in a dining room within abuilding, then the sensitivity of automated detection of foodconsumption can be adjusted. In an example, a food-consumptionmonitoring system can increase the continuity or level of automatic datacollection when a person is in a restaurant, in a grocery store, in akitchen, or in a dining room. In an example, a person's location can beinferred from analysis of visual signals or auditory signals instead ofvia a global positioning system. In an example, a person's location canbe inferred from interaction between a device and local RF beacons orlocal wireless networks.

In an example, a food-consumption monitor or food-identifying sensor canhave a biological component. In an example, a food-identifying sensorcan use biological or biomimetic components to identify specific foods,ingredients, or nutrients. In various examples, a food-identifyingsensor can use one or more biological or biomimetic components selectedfrom the group consisting of: biochemical sensor, antibodies orantibody-based chemical receptor, enzymes or enzyme-based chemicalreceptor, protein or protein-based chemical receptor, biomarker for aspecific dietary nutrient, biomembrane or biomembrane-based sensor,porous polymer or filter paper containing a chemical reagent, nucleicacid-based sensor, polynucleotide-based sensor, artificial taste buds orbiomimetic artificial tongue, and taste bud cells in communication withan electromagnetic sensor.

In an example, a food-consumption monitor or food-identifying sensor canbe a taste or smell sensor. In an example, a sensor can be an artificialtaste bud that emulates the function of a natural taste bud. In anexample, a sensor can be an artificial olfactory receptor that emulatesthe function of a natural olfactory receptor. In an example, a sensorcan comprise biological taste buds or olfactory receptors that areconfigured to be in electrochemical communication with an electronicdevice. In an example, a sensor can be an electronic tongue. In anexample, a sensor can be an electronic nose.

In an example, a food-consumption monitor or food-identifying sensor canbe a high-energy sensor. In an example, a high-energy sensor canidentify a selected type of food, ingredient, or nutrient based on theinteraction of microwaves or x-rays with a portion of food. In variousexamples a high-energy sensor to detect food consumption or identifyconsumption of a selected type of nutrient can be selected from thegroup consisting of: a microwave sensor, a microwave spectrometer, andan x-ray detector.

In an example, a person's consumption of food or the identification of aselected type of food, ingredient, or nutrient can be done by a sensorarray. A sensor array can comprise multiple sensors of different types.In an example, multiple sensors in a sensor array can operatesimultaneously in order to jointly identify food consumption or tojointly identify a selected type of food, ingredient, or nutrient. In anexample, a sensor array can comprise multiple cross-reactive sensors. Inan example, different sensors in a sensor array can operateindependently to identify different types of foods, ingredients, ornutrients. In an example, a single sensor can detect different types offoods, ingredients, or nutrients.

In various examples, a food-consumption monitor or food-identifyingsensor can be selected from the group consisting of: chemical sensor,biochemical sensor, amino acid sensor, chemiresistor, chemoreceptor,photochemical sensor, optical sensor, chromatography sensor, fiber opticsensor, infrared sensor, optoelectronic sensor, spectral analysissensor, spectrophotometer, olfactory sensor, electronic nose, metaloxide semiconductor sensor, conducting polymer sensor, quartz crystalmicrobalance sensor, electromagnetic sensor, variable impedance sensor,variable resistance sensor, conductance sensor, neural impulse sensor,EEG sensor, EGG sensor, EMG sensor, interferometer, galvanic skinresponse sensor, cholesterol sensor, HDL sensor, LDL sensor, electrode,neuroelectrical sensor, neural action potential sensor, Micro ElectricalMechanical System (MEMS) sensor, laboratory-on-a-chip, or medichip,micronutrient sensor, osmolality sensor, protein-based sensor orreagent-based sensor, saturated fat sensor or trans fat sensor, actionpotential sensor, biological sensor, enzyme-based sensor, protein-basedsensor, reagent-based sensor, camera, video camera, fixed focal-lengthcamera, variable focal-length camera, pattern recognition sensor,microfluidic sensor, motion sensor, accelerometer, flow sensor, straingauge, electrogoniometer, inclinometer, peristalsis sensor,multiple-analyte sensor array, an array of cross-reactive sensors, pHlevel sensor, sodium sensor, sonic energy sensor, microphone,sound-based chewing sensor, sound-based swallow sensor, ultrasonicsensor, sugar sensor, glucose sensor, temperature sensor, thermometer,and thermistor.

In an example, a sensor to monitor, detect, or sense food consumption orto identify consumption of a selected type of food, ingredient, ornutrient can be a wearable sensor that is worn by the person whose foodconsumption is monitored, detected, or sensed. In an example, a wearablefood-consumption monitor or food-identifying sensor can be worn directlyon a person's body. In an example a wearable food-consumption monitor orfood-identifying sensor can be worn on, or incorporated into, a person'sclothing.

In various examples, a wearable sensor can be worn on a person in alocation selected from the group consisting of: wrist, neck, finger,hand, head, ear, eyes, nose, teeth, mouth, torso, chest, waist, and leg.In various examples, a wearable sensor can be attached to a person or toa person's clothing by a means selected from the group consisting of:strap, clip, clamp, snap, pin, hook and eye fastener, magnet, andadhesive.

In various examples, a wearable sensor can be worn on a person in amanner like a clothing accessory or piece of jewelry selected from thegroup consisting of: wristwatch, wristphone, wristband, bracelet,cufflink, armband, armlet, and finger ring; necklace, neck chain,pendant, dog tags, locket, amulet, necklace phone, and medallion;eyewear, eyeglasses, spectacles, sunglasses, contact lens, goggles,monocle, and visor; clip, tie clip, pin, brooch, clothing button, andpin-type button; headband, hair pin, headphones, ear phones, hearingaid, earring; and dental appliance, palatal vault attachment, and nosering.

In an example, a sensor to monitor, detect, or sense food consumption orto identify consumption of a selected type of food, ingredient ornutrient can be a utensil-based sensor such as a spoon or fork. In anexample, a utensil-based food-consumption monitor or food-identifyingsensor can be attached to a generic food utensil. In an example, autensil-based sensor can be incorporated into specialized “smartutensil.” In an example, a sensor can be attached to, or incorporatedinto a smart fork or smart spoon. In an example, a sensor can beattached to, or incorporated into, a beverage holding member such as aglass, cup, mug, or can. In an example, a food-identifying sensor can beincorporated into a portable food probe.

In an example, a device to measure a person's consumption of at leastone selected type of food, ingredient, or nutrient can comprise one ormore sensors that are integrated into a place setting. In variousexamples, sensors can be integrated into one or more of the followingplace setting members: plate, glass, cup, bowl, serving dish, place mat,fork, spoon, knife, and smart utensil. In various examples, a placesetting member can incorporate a sensor selected from the groupconsisting of: scale, camera, chemical receptor, spectroscopy sensor,infrared sensor, electromagnetic sensor. In an example, a place settingmember with an integrated food sensor can collect data concerning foodwith which the place setting member is in contact at different times. Inan example, changes in measurements concerning food at different timescan be used to estimate the amount of food that a person is served, theamount of food that a person actually eats, and the amount of left-overfood that a person does not eat.

In an example, a sensor to detect food consumption or to identifyconsumption of a selected type of food, ingredient, or nutrient can beincorporated into a multi-purpose mobile electronic device such as acell phone, mobile phone, smart phone, smart watch, electronic tabletdevice, electronic book reader, electronically-functional jewelry, orother portable consumer electronics device. In an example, a smart phoneapplication can turn the camera function of a smart phone into a meansof food identification. In an example, such a smart phone applicationcan be in wireless communication with a wearable device that is worn bythe person whose food consumption is being measured.

In an example, a wearable device can prompt a person to collectinformation concerning food consumption using a smart phone application.In an example, a wearable device can automatically activate a smartphone or other portable electronic device to collect informationconcerning food consumption. In an example, a wearable device canautomatically trigger a smart phone or other portable electronic deviceto start recording audio information using the smart phone's microphonewhen the wearable device detects that the person is probably eating. Inan example, a wearable device can automatically trigger a smart phone orother portable electronic device to start recording visual informationusing the smart phone's camera when the wearable device detects that theperson is probably eating.

In an example, a food-consumption monitor or specific food-identifyingsensor can monitor, detect, and/or analyze chewing or swallowing actionsby a person. In particular, such a monitor or sensor can differentiatebetween chewing and swallowing actions that are probably associated witheating vs. other activities. In various examples, chewing or swallowingcan be monitored, detected, sensed, or analyzed based on sonic energy(differentiated from speaking, talking, singing, coughing, or othernon-eating sounds), motion (differentiated from speaking or other mouthmotions), imaging (differentiated from other mouth-related activities)or electromagnetic energy (such as electromagnetic signals from mouthmuscles). There are differences in food consumed per chew or per swallowbetween people, and even for the same person over time, based on thetype of food, the person's level of hunger, and other variables. Thiscan make it difficult to estimate the amount of food consumed based onlyon the number of chews or swallows.

In an example, a food-consumption monitor or food-identifying sensor canmonitor a particular body member. In various examples, such a monitor orsensor can be selected from the group consisting of: a blood monitor(for example using a blood pressure monitor, a blood flow monitor, or ablood glucose monitor); a brain monitor (such as anelectroencephalographic monitor); a heart monitor (such aselectrocardiographic monitor, a heartbeat monitor, or a pulse ratemonitor); a mouth function monitor (such as a chewing sensor, a bitingsensor, a jaw motion sensor, a swallowing sensor, or a salivacomposition sensor); a muscle function monitor (such as anelectromyographic monitor or a muscle pressure sensor); a nerve monitoror neural monitor (such as a neural action potential monitor, a neuralimpulse monitor, or a neuroelectrical sensor); a respiration monitor(such as a breathing monitor, an oxygen consumption monitor, an oxygensaturation monitor, a tidal volume sensor, or a spirometry monitor); askin sensor (such as a galvanic skin response monitor, a skinconductance sensor, or a skin impedance sensor); and a stomach monitor(such as an electrogastrographic monitor or a stomach motion monitor).In various examples, a sensor can monitor sonic energy orelectromagnetic energy from selected portions of a person'sgastrointestinal tract (ranging from the mouth to the intestines) orfrom nerves which innervate those portions. In an example, a monitor orsensor can monitor peristaltic motion or other movement of selectedportions of a person's gastrointestinal tract.

In an example, a monitor or sensor to detect food consumption or toidentify a selected type of food, ingredient, or nutrient can be amicro-sampling sensor. In an example, a micro-sampling sensor canautomatically extract and analyze micro-samples of food, intra-oralfluid, saliva, intra-nasal air, chyme, or blood. In an example, amicro-sampling sensor can collect and analyze micro-samplesperiodically. In an example, a micro-sampling sensor can collect andanalyze micro-samples randomly. In an example, a micro-sampling sensorcan collect and analyze micro-samples when a different sensor indicatesthat a person is probably consuming food. In an example, amicro-sampling sensor can be selected from the group consisting of:microfluidic sampling system, microfluidic sensor array, and micropump.

In an example, a sensor to detect food consumption and/or identifyconsumption of a selected type of food, ingredient, or nutrient canincorporate microscale or nanoscale technology. In various examples, asensor to detect food consumption or identify a specific food,ingredient, or nutrient can be selected from the group consisting of:micro-cantilever sensor, microchip sensor, microfluidic sensor,nano-cantilever sensor, nanotechnology sensor, Micro ElectricalMechanical System (MEMS) sensor, laboratory-on-a-chip, and medichip.

5. Smart Watch or Other Wearable Component

In an example, a food-consumption monitor or food-identifying sensor canbe incorporated into a smart watch or other device that is worn on aperson's wrist. In an example, a food-consumption monitor orfood-identifying sensor can be worn on, or attached to, other members ofa person's body or to a person's clothing. In an example, a device formeasuring a person's consumption of at least one selected type of food,ingredient, or nutrient can be worn on, or attached to, a person's bodyor clothing. In an example, a device can be worn on, or attached to, apart of a person's body that is selected from the group consisting of:wrist (one or both), hand (one or both), or finger; neck or throat; eyes(directly such as via contact lens or indirectly such as via eyewear);mouth, jaw, lips, tongue, teeth, or upper palate; arm (one or both);waist, abdomen, or torso; nose; ear; head or hair; and ankle or leg.

In an example, a device for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient can be worn in amanner similar to a piece ofjewelry or accessory. In various examples, afood consumption measuring device can be worn in a manner similar to apiece of or accessory selected from the group consisting of: smartwatch, wrist band, wrist phone, wrist watch, fitness watch, or otherwrist-worn device; finger ring or artificial finger nail; arm band, armbracelet, charm bracelet, or smart bracelet; smart necklace, neck chain,neck band, or neck-worn pendant; smart eyewear, smart glasses,electronically-functional eyewear, virtual reality eyewear, orelectronically-functional contact lens; cap, hat, visor, helmet, orgoggles; smart button, brooch, ornamental pin, clip, smart beads;pin-type, clip-on, or magnetic button; shirt, blouse, jacket, coat, ordress button; head phones, ear phones, hearing aid, ear plug, orear-worn bluetooth device; dental appliance, dental insert, upper palateattachment or implant; tongue ring, ear ring, or nose ring;electronically-functional skin patch and/or adhesive patch; undergarmentwith electronic sensors; head band, hair band, or hair clip; ankle strapor bracelet; belt or belt buckle; and key chain or key ring.

In an example, a device for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient can be incorporatedor integrated into an article of clothing or a clothing-relatedaccessory. In various examples, a device for measuring food consumptioncan be incorporated or integrated into one of the following articles ofclothing or clothing-related accessories: belt or belt buckle; neck tie;shirt or blouse; shoes or boots; underwear, underpants, briefs,undershirt, or bra; cap, hat, or hood; coat, jacket, or suit; dress orskirt; pants, jeans, or shorts; purse; socks; and sweat suit.

In an example, a device for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient can be attached to aperson's body or clothing. In an example, a device to measure foodconsumption can be attached to a person's body or clothing using anattachment means selected from the group consisting of: band, strap,chain, hook and eye fabric, ring, adhesive, bracelet, buckle, button,clamp, clip, elastic band, eyewear, magnet, necklace, piercing, pin,string, suture, tensile member, wrist band, and zipper. In an example, adevice can be incorporated into the creation of a specific article ofclothing. In an example, a device to measure food consumption can beintegrated into a specific article of clothing by a means selected fromthe group consisting of: adhesive, band, buckle, button, clip, elasticband, hook and eye fabric, magnet, pin, pocket, pouch, sewing, strap,tensile member, and zipper.

In an example, a wearable device for measuring a person's consumption ofat least one selected type of food, ingredient, or nutrient can compriseone or more sensors selected from the group consisting of: motionsensor, accelerometer (single multiple axis), electrogoniometer, orstrain gauge; optical sensor, miniature still picture camera, miniaturevideo camera, miniature spectroscopy sensor; sound sensor, miniaturemicrophone, speech recognition software, pulse sensor, ultrasoundsensor; electromagnetic sensor, skin galvanic response (Galvanic SkinResponse) sensor, EMG sensor, chewing sensor, swallowing sensor;temperature sensor, thermometer, infrared sensor; and chemical sensor,chemical sensor array, miniature spectroscopy sensor, glucose sensor,cholesterol sensor, or sodium sensor.

In an example, a device and system for measuring a person's consumptionof at least one selected type of food, ingredient, or nutrient can beentirely wearable or include a wearable component. In an example, awearable device or component can be worn directly on a person's body,can be worn on a person's clothing, or can be integrated into a specificarticle of clothing. In an example, a wearable device for measuring foodconsumption can be in wireless communication with an external device. Invarious examples, a wearable device for measuring food consumption canbe in wireless communication with an external device selected from thegroup consisting of: a cell phone, an electronic tablet,electronically-functional eyewear, a home electronics portal, aninternet portal, a laptop computer, a mobile phone, a remote computer, aremote control unit, a smart phone, a smart utensil, a television set,and a virtual menu system.

In an example, a wearable device for measuring a person's consumption ofat least one selected type of food, ingredient, or nutrient can comprisemultiple components selected from the group consisting of: CentralProcessing Unit (CPU) or microprocessor; food-consumption monitoringcomponent (motion sensor, electromagnetic sensor, optical sensor, and/orchemical sensor); graphic display component (display screen and/orcoherent light projection); human-to-computer communication component(speech recognition, touch screen, keypad or buttons, and/or gesturerecognition); memory component (flash, RAM, or ROM); power source and/orpower-transducing component; time keeping and display component;wireless data transmission and reception component; and strap or band.

6. Smart Utensil, Mobile Phone, or Other Hand-Held Component

In an example, a device, method, and system for measuring consumption ofselected types of foods, ingredients, or nutrients can include ahand-held component in addition to a wearable component. In an example,a hand-held component can be linked or combined with a wearablecomponent to form an integrated system for measuring a person'sconsumption of at least one selected type of food, ingredient, ornutrient. In an example, the combination and integration of a wearablemember and a hand-held member can provide advantages that are notpossible with either a wearable member alone or a hand-held memberalone. In an example, a wearable member of such a system can be afood-consumption monitor. In an example, a hand-held member of such asystem can be a food-identifying sensor.

In an example, a wearable member can continually monitor to detect whenthe person is consuming food, wherein this continual monitoring does notsignificantly intrude on the person's privacy. In an example, ahand-held member may be potentially more intrusive with respect toprivacy when it operates, but is only activated to operate when foodconsumption is detected by the wearable member. In an example, wearableand hand-held components of such a system can be linked by wirelesscommunication. In an example, wearable and held-held components of sucha system can be physically linked by a flexible wire. In an example, ahand-held component can be removably attached to the wearable member anddetached for use in identifying at least one selected type of food,ingredient, or nutrient.

In an example, a hand-held component of a device or system for measuringa person's consumption of at least one selected type of food,ingredient, or nutrient can be a hand-held smart food utensil or foodprobe. In an example, a hand-held component of a device or system formeasuring a person's consumption of at least one selected type of food,ingredient, or nutrient can be a hand-held mobile phone or other generalconsumer electronics device that performs multiple functions. In anexample, a mobile phone application can link or integrate the operationof the mobile phone with the operation of a wearable component of asystem for measuring a person's consumption of at least one selectedtype of food, ingredient, or nutrient.

In various examples, a hand-held component can be selected from thegroup consisting of: smart utensil, smart spoon, smart fork, smart foodprobe, smart bowl, smart chop stick, smart dish, smart glass, smartplate, electronically-functional utensil, electronically-functionalspoon, electronically-functional fork, electronically-functional foodprobe, electronically-functional bowl, electronically-functional chopstick, electronically-functional dish, electronically-functional glass,electronically-functional plate, smart phone, mobile phone, cell phone,electronic tablet, and digital camera.

In various examples, a food-consumption monitoring and nutrientidentifying system can comprise a combination of a wearable componentand a hand-held component that is selected from the group consisting of:smart watch and smart food utensil; smart watch and food probe; smartwatch and mobile phone; smart watch and electronic tablet; smart watchand digital camera; smart bracelet and smart food utensil; smartbracelet and food probe; smart bracelet and mobile phone; smart braceletand electronic tablet; smart bracelet and digital camera; smart necklaceand smart food utensil; smart necklace and food probe; smart necklaceand mobile phone; smart necklace and electronic tablet; and smartnecklace and digital camera.

In an example, a wearable food-consumption monitor (such as may beembodied in a smart watch, smart bracelet, or smart necklace) and ahand-held food-identifying sensor (such as may be embodied in a smartutensil, food probe, or smart phone) can be linked or combined togetherinto an integrated system for measuring a person's consumption of atleast one selected type of food, ingredient, or nutrient. In an example,wearable and held-held components such a system can be separatecomponents that are linked by wireless communication. In an example,wearable and held-held components of such a system can be physicallyconnected by a flexible element. In an example, wearable and hand-heldcomponents can be physically attached or detached for use. In anexample, a hand-held component can be a removable part of a wearablecomponent for easier portability and increased user compliance for alleating events. In an example, a smart utensil or food probe can beremoved from a wearable component to identify food prior to, or duringconsumption. This can increase ease of use and user compliance with foodidentification for all eating events.

A smart food utensil can be a food utensil that is specifically designedto help measure a person's consumption of at least one selected type offood, ingredient, or nutrient. In an example, a smart utensil can be afood utensil that is equipped with electronic and/or sensoryfunctionality. In an example, a smart food utensil can be designed tofunction like a regular food utensil, but is also enhanced with sensorsin order to detect food consumption and/or identify consumption ofselected types of foods, ingredients, or nutrients.

A regular food utensil can be narrowly defined as a tool that iscommonly used to convey a single mouthful of food up to a person'smouth. In this narrow definition, a food utensil can be selected fromthe group consisting of: fork, spoon, spork, and chopstick. In anexample, a food utensil can be more broadly defined as a tool that isused to apportion food into mouthfuls during food consumption or toconvey a single mouthful of food up to a person's mouth during foodconsumption. This broader definition includes cutlery and knives used atthe time of food consumption in addition to forks, spoons, sporks, andchopsticks.

In an example, a food utensil may be more broadly defined to alsoinclude tools and members that are used to convey amounts of food thatare larger than a single mouthful and to apportion food into servingsprior to food consumption by an individual. Broadly defined in such amanner, a food utensil can be selected from the group consisting of:fork, spoon, spork, knife, chopstick, glass, cup, mug, straw, can,tablespoon, teaspoon, ladle, scoop, spatula, tongs, dish, bowl, andplate. In an example, a smart utensil is an electronically-functionalutensil. In an example, a smart utensil can be a utensil with one orbuilt-in functions selected from the group consisting of: detecting useto convey food; detecting food consumption; measuring the speed, rate,or pace of food consumption; measuring the amount of food consumed;identifying the type of food consumed; and communicating informationconcerning food consumption to other devices or system components.

In an example, a food-consumption monitor or food-identifying sensor canbe incorporated into, or attached to, a food utensil. In an example,such a sensor can be an integral part of a specialized smart utensilthat is specifically designed to measure food consumption or detectconsumption of at least one selected type of food, ingredient, ornutrient. In an example, such a sensor can be designed to be removablyattached to a generic food utensil so that any generic utensil can beused. In an example, a sensor can be attached to a generic utensil bytension, a clip, an elastic band, magnetism, or adhesive.

In an example, such a sensor, or a smart utensil of which this sensor isa part, can be in wireless communication with a smart watch or othermember that is worn on a person's wrist, hand, or arm. In this manner, asystem or device can tell if a person is using the smart utensil whenthey eat based on the relative movements and/or proximity of a smartutensil to a smart watch. In an example, a smart utensil can be acomponent of a multi-component system to measure of person's consumptionof at least one selected type of food, ingredient, or nutrient.

In an example, a smart food utensil or food probe can identify the typesand amounts of consumed foods, ingredients, or nutrients by being inoptical communication with food. In an example, a smart food utensil canidentify the types and amounts of food consumed by taking pictures offood. In an example, a smart food utensil can take pictures of food thatis within a reachable distance of a person. In an example, a smart foodutensil can take pictures of food on a plate. In an example, a smartfood utensil can take pictures of a portion of food as that food isconveyed to a person's mouth via the utensil.

In an example, a smart food utensil can identify the type of food byoptically analyzing food being consumed. In an example, a smart foodutensil can identify the types and amounts of food consumed by recordingthe effects light that is interacted with food. In an example, a smartfood utensil can identify the types and amounts of food consumed viaspectroscopy. In an example, a smart food utensil can performspectroscopic analysis of a portion of food as that food is conveyed toa person's mouth via the utensil. In an example, a smart food utensilcan measure the amount of food consumed using a photo-detector.

In an example, a smart food utensil or food probe can identify the typesand amounts of consumed foods, ingredients, or nutrients by performingchemical analysis of food. In an example, a smart food utensil canidentify the types and amounts of food consumed by performing chemicalanalysis of the chemical composition of food. In an example, a smartfood utensil can collect data that is used to analyze the chemicalcomposition of food by direct contact with food. In an example, a smartfood utensil can identify the type of food, ingredient, or nutrientbeing consumed by being in fluid or gaseous communication with food. Inan example, a smart food utensil can include an array of chemicalsensors with which a sample of food interacts.

In an example, a smart food utensil can collect data that is used toanalyze the chemical composition of food by measuring the absorption oflight, sound, or electromagnetic energy by food that is in proximity tothe person whose consumption is being monitored. In an example, a smartfood utensil can collect data that is used to analyze the chemicalcomposition of food by measuring the reflection of light, sound, orelectromagnetic energy by food that is in proximity to the person whoseconsumption is being monitored. In an example, a smart food utensil cancollect data that is used to analyze the chemical composition of food bymeasuring the reflection of different wavelengths of light, sound, orelectromagnetic energy by food that is in proximity to the person whoseconsumption is being monitored.

In an example, a smart food utensil can identify the types and amountsof food consumed by measuring the effects of interacting food withelectromagnetic energy. In an example, a smart food utensil can estimatethe amount of food that a person consumes by tracking utensil motionswith an accelerometer. In various examples, one or more sensors that arepart of, or attached to, a smart food utensil can be selected from thegroup consisting of: motion sensor, accelerometer, strain gauge,inertial sensor, scale, weight sensor, or pressure sensor; miniaturecamera, video camera, optical sensor, optoelectronic sensor,spectrometer, spectroscopy sensor, or infrared sensor; chemical sensor,chemical receptor array, or spectroscopy sensor; microphone, soundsensor, or ultrasonic sensor; and electromagnetic sensor, capacitivesensor, inductance sensor, or piezoelectric sensor.

In an example, a wearable member (such as a smart watch) can continuallymonitor for possible food consumption, but a smart utensil is only usedwhen the person is eating. In an example, a device or system formeasuring food consumption can compare the motion of a smart utensilwith the motion of a wearable member (such as a smart watch) in order todetect whether the smart utensil is being properly used whenever theperson is eating food. In an example, a device or system for measuringfood consumption can track the movement of a smart utensil that a personshould use consistently to eat food, track the movement of a wearablemotion sensor (such as a smart watch) that a person wears continuously,and compare the movements to determine whether the person always usesthe smart utensil to eat. In an example, this device or system canprompt the person to use the smart utensil when comparison of the motionof the smart utensil with the motion of a wearable motion sensor (suchas a smart watch) indicates that the person is not using the smartutensil when they are eating.

In an example, a device or system for measuring food consumption canmonitor the proximity of a smart utensil to a wearable member (such as asmart watch) in order to detect whether the smart utensil is beingproperly used whenever the person is eating food. In an example, thisdevice or system can prompt the person to use the smart utensil whenlack of proximity between the smart utensil and a wearable member (suchas a smart watch) indicates that the person is not using the smartutensil when they are eating. In an example, a device or system formeasuring food consumption can detect if a smart utensil is attached to,or near to, a smart watch. In an example, a device or system formeasuring food consumption can prompt a person to use a smart utensil ifthe smart utensil is not attached to, or near to, a smart watch when theperson is eating.

In an example, a food-consumption monitoring and nutrient identifyingsystem can include a hand-held component that is selected from the groupconsisting of: smart phone, mobile phone, cell phone, holophone, orapplication of such a phone; electronic tablet, other flat-surfacemobile electronic device, Personal Digital Assistant (PDA), or laptop;digital camera; and smart eyewear, electronically-functional eyewear, oraugmented reality eyewear. In an example, such a hand-held component canbe in wireless communication with a wearable component of such a system.In an example, a device, method, or system for detecting foodconsumption or measuring consumption of a selected type of food,ingredient, or nutrient can include integration with a general-purposemobile device that is used to collects data concerning food consumption.In an example, the hand-held component of such a system can be a generalpurpose device, of which collecting data for food identification is onlyone among many functions that it performs. In an example, a system formeasuring a person's consumption of at least one selected type of food,ingredient, or nutrient can comprise: a wearable member that continuallymonitors for possible food consumption; a hand-held smart phone that isused to take pictures of food that will be consumed; wirelesscommunication between the wearable member and the smart phone; andsoftware that integrates the operation of the wearable member and thesmart phone.

In an example, the hand-held component of a food-consumption monitoringand nutrient identifying system can be a general purpose smart phonewhich collects information concerning food by taking pictures of food.In an example, this smart phone can be in wireless communication with awearable component of the system, such as a smart watch. In an example,the hand-held component of such a system must be brought into physicalproximity with food that will be consumed in order to measure theresults of interaction between food and light, sound, or electromagneticenergy.

In an example, a hand-held component of such a system requires voluntaryaction by a person in order to collect data for food identification inassociation with each eating event apart from the actual act of eating.In an example, a mobile phone must be pointed toward food by a personand triggered to take pictures of that food. In an example, a hand-heldcomponent of such a system must be brought into fluid or gaseouscommunication with food in order to chemically analyze the compositionof food. In an example, a wearable member (such as a smart watch) cancontinually monitor for possible food consumption, but a smart phone isonly used for food identification when the person is eating. In anexample, this device or system can prompt the person to use a smartphone for food identification when the person is eating.

In an example, a smart phone can identify the types and amounts ofconsumed foods, ingredients, or nutrients by being in opticalcommunication with food. In an example, a smart phone can collectinformation for identifying the types and amounts of food consumed bytaking pictures of food. In an example, a smart phone can take picturesof food that is within a reachable distance of a person. In an example,a smart phone can take pictures of food on a plate.

In an example, a smart phone can collect data that is used to analyzethe chemical composition of food by measuring the absorption of light,sound, or electromagnetic energy by food that is in proximity to theperson whose consumption is being monitored. In an example, a smartphone can collect data that is used to analyze the chemical compositionof food by measuring the reflection of different wavelengths of light,sound, or electromagnetic energy by food that is in proximity to theperson whose consumption is being monitored. In various examples, one ormore sensors that are part of, or attached to, a smart phone can beselected from the group consisting of: miniature camera, video camera,optical sensor, optoelectronic sensor, spectrometer, spectroscopysensor, and infrared sensor.

7. User Interface

In an example, a device for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient can include ahuman-to-computer interface for communication from a human to acomputer. In various examples, a device for measuring a person'sconsumption of at least one selected type of food, ingredient, ornutrient can include a human-to-computer interface selected from thegroup consisting of: speech recognition or voice recognition interface;touch screen or touch pad; physical keypad/keyboard, virtual keypad orkeyboard, control buttons, or knobs; gesture recognition interface orholographic interface; motion recognition clothing; eye movementdetector, smart eyewear, and/or electronically-functional eyewear; headmovement tracker; conventional flat-surface mouse, 3D blob mouse, trackball, or electronic stylus; graphical user interface, drop down menu,pop-up menu, or search box; and neural interface or EMG sensor.

In an example, such a human-to-computer interface can enable a user todirectly enter information concerning food consumption. In an example,such direct communication of information can occur prior to foodconsumption, during food consumption, and/or after food consumption. Inan example, such a human-to-computer interface can enable a user toindirectly collect information concerning food consumption. In anexample, such indirect collection of information can occur prior to foodconsumption, during food consumption, and/or after food consumption.

In an example, a device for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient can include acomputer-to-human interface for communication from a computer to ahuman. In an example, a device and method for monitoring and measuring aperson's food consumption can provide feedback to the person. In anexample, a computer-to-human interface can communicate information aboutthe types and amounts of food that a person has consumed, shouldconsume, or should not consume. In an example, a computer-to-humaninterface can provide feedback to a person concerning their eatinghabits and the effects of those eating habits. In an example, thisfeedback can prompt the person to collect more information concerningthe types and amounts of food that the person is consuming. In anexample, a computer-to-human interface can be used to not just provideinformation concerning eating behavior, but also to change eatingbehavior.

In various examples, a device for measuring a person's consumption of atleast one selected type of food, ingredient, or nutrient can providefeedback to the person that is selected from the group consisting of:auditory feedback (such as a voice message, alarm, buzzer, ring tone, orsong); feedback via computer-generated speech; mild external electriccharge or neural stimulation; periodic feedback at a selected time ofthe day or week; phantom taste or smell; phone call; pre-recorded audioor video message by the person from an earlier time; television-basedmessages; and tactile, vibratory, or pressure-based feedback.

In various examples, a device for measuring a person's consumption of atleast one selected type of food, ingredient, or nutrient can providefeedback to the person that is selected from the group consisting of:feedback concerning food consumption (such as types and amounts offoods, ingredients, and nutrients consumed, calories consumed, caloriesexpended, and net energy balance during a period of time); informationabout good or bad ingredients in nearby food; information concerningfinancial incentives or penalties associated with acts of foodconsumption and achievement of health-related goals; informationconcerning progress toward meeting a weight, energy-balance, and/orother health-related goal; information concerning the calories ornutritional components of specific food items; and number of caloriesconsumed per eating event or time period.

In various examples, a device for measuring a person's consumption of atleast one selected type of food, ingredient, or nutrient can providefeedback to the person that is selected from the group consisting of:augmented reality feedback (such as virtual visual elements superimposedon foods within a person's field of vision); changes in a picture orimage of a person reflecting the likely effects of a continued patternof food consumption; display of a person's progress toward achievingenergy balance, weight management, dietary, or other health-relatedgoals; graphical display of foods, ingredients, or nutrients consumedrelative to standard amounts (such as embodied in pie charts, barcharts, percentages, color spectrums, icons, emoticons, animations, andmorphed images); graphical representations of food items; graphicalrepresentations of the effects of eating particular foods; holographicdisplay; information on a computer display screen (such as a graphicaluser interface); lights, pictures, images, or other optical feedback;touch screen display; and visual feedback throughelectronically-functional eyewear.

In various examples, a device for measuring a person's consumption of atleast one selected type of food, ingredient, or nutrient can providefeedback to the person that is selected from the group consisting of:advice concerning consumption of specific foods or suggested foodalternatives (such as advice from a dietician, nutritionist, nurse,physician, health coach, other health care professional, virtual agent,or health plan); electronic verbal or written feedback (such as phonecalls, electronic verbal messages, or electronic text messages); livecommunication from a health care professional; questions to the personthat are directed toward better measurement or modification of foodconsumption; real-time advice concerning whether to eat specific foodsand suggestions for alternatives if foods are not healthy; socialfeedback (such as encouragement or admonitions from friends and/or asocial network); suggestions for meal planning and food consumption foran upcoming day; and suggestions for physical activity and caloricexpenditure to achieve desired energy balance outcomes.

8. Power Source and Wireless Communication

In an example, a wearable and/or hand-held member of a device formeasuring a person's consumption of at least one selected type of food,ingredient, or nutrient can comprise multiple components selected fromthe group consisting of: a food-consumption monitor or food-identifyingsensor; a central processing unit (CPU) such as a microprocessor; adatabase of different types of food and food attributes; a memory tostore, record, and retrieve data such as the cumulative amount consumedfor at least one selected type of food, ingredient, or nutrient; acommunications member to transmit data to from external sources and toreceive data from external sources; a power source such as a battery orpower transducer; a human-to-computer interface such as a touch screen,keypad, or voice recognition interface; and a computer-to-humaninterface such as a display screen or voice-producing interface.

In an example, the power source for a wearable and/or hand-held memberof a device for measuring a person's consumption of at least oneselected type of food, ingredient, or nutrient can be selected from thegroup consisting of: power from a power source that is internal to thedevice during regular operation (such as an internal battery, capacitor,energy-storing microchip, or wound coil or spring); power that isobtained, harvested, or transduced from a power source other than theperson's body that is external to the device (such as a rechargeablebattery, electromagnetic inductance from external source, solar energy,indoor lighting energy, wired connection to an external power source,ambient or localized radiofrequency energy, or ambient thermal energy);and power that is obtained, harvested, or transduced from the person'sbody (such as kinetic or mechanical energy from body motion,electromagnetic energy from the person's body, blood flow or otherinternal fluid flow, glucose metabolism, or thermal energy from theperson's body.

In an example, a device or system for measuring a person's consumptionof at least one selected type of food, ingredient, or nutrient caninclude one or more communications components for wireless transmissionand reception of data. In an example, multiple communications componentscan enable wireless communication (including data exchange) betweenseparate components of such a device and system. In an example, acommunications component can enable wireless communication with anexternal device or system. In various examples, the means of thiswireless communication can be selected from the group consisting of:radio transmission, Bluetooth transmission, Wi-Fi, and infrared energy.

In various examples, a device and system for measuring food consumptioncan be in wireless communication with an external device or systemselected from the group consisting of: internet portal; smart phone,mobile phone, cell phone, holophone, or application of such a phone;electronic tablet, other flat-surface mobile electronic device, PersonalDigital Assistant (PDA), remote control unit, or laptop; smart eyewear,electronically-functional eyewear, or augmented reality eyewear;electronic store display, electronic restaurant menu, or vendingmachine; and desktop computer, television, or mainframe computer. Invarious examples, a device can receive food-identifying information froma source selected from the group consisting of: electromagnetictransmissions from a food display or RFID food tag in a grocery store,electromagnetic transmissions from a physical menu or virtual userinterface at a restaurant, and electromagnetic transmissions from avending machine.

In an example, data concerning food consumption that is collected by awearable or hand-held device can be analyzed by a data processing unitwithin the device in order to identify the types and amounts of foods,ingredients, or nutrients that a person consumes. In an example, dataconcerning food consumption that is collected by a smart watch can beanalyzed within the housing of the watch. In an example, data concerningfood consumption that is collected by a smart food utensil can beanalyzed within the housing of the utensil.

In another example, data concerning food consumption that is collectedby a wearable or hand-held device can be transmitted to an externaldevice or system for analysis at a remote location. In an example,pictures of food can be transmitted to an external device or system forfood identification at a remote location. In an example, chemicalanalysis results can be transmitted to an external device or system forfood identification at a remote location. In an example, the results ofanalysis at a remote location can be transmitted back to a wearable orhand-held device.

9. Automatic Food Identification

In an example, a device for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient can identify andtrack the selected types and amounts of foods, ingredients, or nutrientsthat the person consumes in an entirely automatic manner. In an example,such identification can occur in a partially automatic manner in whichthere is interaction between automated and human identification methods.

In an example, a device for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient can identify andtrack food consumption at the point of selection or point of sale. In anexample, a device for monitoring food consumption or consumption ofselected types of foods, ingredients, or nutrients can approximate suchmeasurements by tracking a person's food selections and purchases at agrocery store, at a restaurant, or via a vending machine. Trackingpurchases can be relatively easy to do, since financial transactions arealready well-supported by existing information technology. In anexample, such tracking can be done with specific methods of payment,such as a credit card or bank account. In an example, such tracking canbe done with electronically-functional food identification means such asbar codes, RFID tags, or electronically-functional restaurant menus.Electronic communication for food identification can also occur betweena food-consumption monitoring device and a vending machine.

In an example, a device for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient can identify foodusing information from a food's packaging or container. In an example,this information can be detected optically by means of a picture oroptical scanner. In an example, food can be identified directly byautomated optical recognition of information on food packaging, such asa logo, label, or barcode. In various examples, optical information on afood's packaging or container that is used to identify the type and/oramount of food can be selected from the group consisting of: bar code,food logo, food trademark design, nutritional label, optical textrecognition, and UPC code. With respect to meals ordered at restaurants,some restaurants (especially fast-food restaurants) have standardizedmenu items with standardized food ingredients. In such cases,identification of types and amounts of food, ingredients, or nutrientscan be conveyed at the point of ordering (via anelectronically-functional menu) or purchase (via purchase transaction).In an example, food can be identified directly by wireless informationreceived from a food display, RFID tag, electronically-functionalrestaurant menu, or vending machine. In an example, food or itsnutritional composition can be identified directly by wirelesstransmission of information from a food display, menu, food vendingmachine, food dispenser, or other point of food selection or sale and adevice that is worn, held, or otherwise transported with a person.

However, there are limitations to estimating food consumption based onfood selections or purchases in a store or restaurant. First, a personmight not eat everything that they purchase through venues that aretracked by the system. The person might purchase food that is eaten bytheir family or other people and might throw out some of the food thatthey purchase. Second, a person might eat food that they do not purchasethrough venues that are tracked by the system. The person might purchasesome food with cash or in venues that are otherwise not tracked. Theperson might eat food that someone else bought, as when eating as aguest or family member. Third, timing differences between when a personbuys food and when they eat it, especially for non-perishable foods, canconfound efforts to associate caloric intake with caloric expenditure tomanage energy balance during a defined period of time. For thesereasons, a robust device for measuring food consumption should (also) beable to identify food at the point of consumption.

In an example, a device, method, or system for measuring a person'sconsumption of at least one selected type of food, ingredient, ornutrient can identify and track a person's food consumption at the pointof consumption. In an example, such a device, method, or system caninclude a database of different types of food. In an example, such adevice, method, or system can be in wireless communication with anexternally-located database of different types of food. In an example,such a database of different types of food and their associatedattributes can be used to help identify selected types of foods,ingredients, or nutrients. In an example, a database of attributes fordifferent types of food can be used to associate types and amounts ofspecific ingredients, nutrients, and/or calories with selected types andamounts of food.

In an example, such a database of different types of foods can includeone or more elements selected from the group consisting of: food color,food name, food packaging bar code or nutritional label, food packagingor logo pattern, food picture (individually or in combinations withother foods), food shape, food texture, food type, common geographic orintra-building locations for serving or consumption, common orstandardized ingredients (per serving, per volume, or per weight),common or standardized nutrients (per serving, per volume, or perweight), common or standardized size (per serving), common orstandardized number of calories (per serving, per volume, or perweight), common times or special events for serving or consumption, andcommonly associated or jointly-served foods.

In an example, a picture of a meal as a whole can be automaticallysegmented into portions of different types of food for comparison withdifferent types of food in a food database. In an example, theboundaries between different types of food in a picture of a meal can beautomatically determined to segment the meal into different food typesbefore comparison with pictures in a food database. In an example, apicture of a meal with multiple types of food can be compared as a wholewith pictures of meals with multiple types of food in a food database.In an example, a picture of a food or a meal comprising multiple typesof food can be compared directly with pictures of food in a fooddatabase.

In an example, a picture of food or a meal comprising multiple types offood can be adjusted, normalized, or standardized before it is comparedwith pictures of food in a food database. In an example, food color canbe adjusted, normalized, or standardized before comparison with picturesin a food database. In an example, food size or scale can be adjusted,normalized, or standardized before comparison with pictures in a fooddatabase. In an example, food texture can be adjusted, normalized, orstandardized before comparison with pictures in a food database. In anexample, food lighting or shading can be adjusted, normalized, orstandardized before comparison with pictures in a food database.

In an example, a food database can be used to identify the amount ofcalories that are associated with an indentified type and amount offood. In an example, a food database can be used to identify the typeand amount of at least one selected type of food that a person consumes.In an example, a food database can be used to identify the type andamount of at least one selected type of ingredient that is associatedwith an identified type and amount of food. In an example, a fooddatabase can be used to identify the type and amount of at least oneselected type of nutrient that is associated with an identified type andamount of food. In an example, an ingredient or nutrient can beassociated with a type of food on a per-portion, per-volume, orper-weight basis.

In an example, a vector of food characteristics can be extracted from apicture of food and compared with a database of such vectors for commonfoods. In an example, analysis of data concerning food consumption caninclude comparison of food consumption parameters between a specificperson and a reference population. In an example, data analysis caninclude analysis of a person's food consumption patterns over time. Inan example, such analysis can track the cumulative amount of at leastone selected type of food, ingredient, or nutrient that a personconsumes during a selected period of time.

In various examples, data concerning food consumption can be analyzed toidentify and track consumption of selected types and amounts of foods,ingredients, or nutrient consumed using one or more methods selectedfrom the group consisting of: linear regression and/or multivariatelinear regression, logistic regression and/or probit analysis, Fouriertransformation and/or fast Fourier transform (FFT), linear discriminantanalysis, non-linear programming, analysis of variance, chi-squaredanalysis, cluster analysis, energy balance tracking, factor analysis,principal components analysis, survival analysis, time series analysis,volumetric modeling, neural network and machine learning.

In an example, a device for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient can identify thetypes and amounts of food consumed in an automated manner based onimages of that food. In various examples, food pictures can be analyzedfor automated food identification using methods selected from the groupconsisting of: image attribute adjustment or normalization; inter-foodboundary determination and food portion segmentation; image patternrecognition and comparison with images in a food database to identifyfood type; comparison of a vector of food characteristics with adatabase of such characteristics for different types of food; scaledetermination based on a fiduciary marker and/or three-dimensionalmodeling to estimate food quantity; and association of selected typesand amounts of ingredients or nutrients with selected types and amountsof food portions based on a food database that links common types andamounts of foods with common types and amounts of ingredients ornutrients. In an example, automated identification of selected types offood based on images and/or automated association of selected types ofingredients or nutrients with that food can occur within a wearable orhand-held device. In an example, data collected by a wearable orhand-held device can be transmitted to an external device whereautomated identification occurs and the results can then be transmittedback to the wearable or hand-held device.

In an example, a device and system for measuring a person's consumptionof at least one selected type of food, ingredient, or nutrient can takepictures of food using a digital camera. In an example, a device andsystem for measuring a person's consumption of at least one selectedtype of food, ingredient, or nutrient can take pictures of food using animaging device selected from the group consisting of: smart watch, smartbracelet, fitness watch, fitness bracelet, watch phone, bracelet phone,wrist band, or other wrist-worn device; arm bracelet; and smart ring orfinger ring. In an example, a device and system for measuring a person'sconsumption of at least one selected type of food, ingredient, ornutrient can take pictures of food using an imaging device selected fromthe group consisting of: smart phone, mobile phone, cell phone,holophone, and electronic tablet.

In an example, a device and system for measuring a person's consumptionof at least one selected type of food, ingredient, or nutrient can takepictures of food using an imaging device selected from the groupconsisting of: smart glasses, visor, or other eyewear;electronically-functional glasses, visor, or other eyewear; augmentedreality glasses, visor, or other eyewear; virtual reality glasses,visor, or other eyewear; and electronically-functional contact lens. Inan example, a device and system for measuring a person's consumption ofat least one selected type of food, ingredient, or nutrient can takepictures of food using an imaging device selected from the groupconsisting of: smart utensil, fork, spoon, food probe, plate, dish, orglass; and electronically-functional utensil, fork, spoon, food probe,plate, dish, or glass. In an example, a device and system for measuringa person's consumption of at least one selected type of food,ingredient, or nutrient can take pictures of food using an imagingdevice selected from the group consisting of: smart necklace, smartbeads, smart button, neck chain, and neck pendant.

In an example, an imaging device can take multiple still pictures ormoving video pictures of food. In an example, an imaging device can takemultiple pictures of food from different angles in order to performthree-dimensional analysis or modeling of the food to better determinethe volume of food. In an example, an imaging device can take multiplepictures of food from different angles in order to better control fordifferences in lighting and portions of food that are obscured from someperspectives. In an example, an imaging device can take multiplepictures of food from different angles in order to performthree-dimensional modeling or volumetric analysis to determine thethree-dimensional volume of food in the picture. In an example, animaging device can take multiple pictures of food at different times,such as before and after an eating event, in order to better determinehow much food the person actually ate (as compared to the amount of foodserved). In an example, changes in the volume of food in sequentialpictures before and after consumption can be compared to the cumulativevolume of food conveyed to a person's mouth by a smart utensil todetermine a more accurate estimate of food volume consumed. In variousexamples, a person can be prompted by a device to take pictures of foodfrom different angles or at different times.

In an example, a device that indentifies a person's food consumptionbased on images of food can receive food images from an imagingcomponent or other imaging device that the person holds in their hand tooperate. In an example, a device that indentifies a person's foodconsumption based on images of food can receive food images from animaging component or other imaging device that the person wears on theirbody or clothing. In an example, a wearable imaging device can be wornin a relatively fixed position on a person's neck or torso so that italways views the space in front of a person. In an example, a wearableimaging device can be worn on a person's wrist, arm, or finger so thatthe field of vision of the device moves as the person moves their arm,wrist, and/or fingers. In an example, a device with a moving field ofvision can monitor both hand-to-food interaction and hand-to-mouthinteraction as the person moves their arm, wrist, and/or hand. In anexample, a wearable imaging device can comprise a smart watch with aminiature camera that monitors the space near a person's hands forpossible hand-to-food interaction and monitors the near a person's mouthfor hand-to-mouth interaction.

In an example, selected attributes or parameters of a food image can beadjusted, standardized, or normalized before the food image is comparedto images in a database of food images or otherwise analyzed foridentifying the type of food. In various examples, these imageattributes or parameters can be selected from the group consisting of:food color, food texture, scale, image resolution, image brightness, andlight angle.

In an example, a device and system for identifying types and amounts offood consumed based on food images can include the step of automaticallysegmenting regions of a food image into different types or portions offood. In an example, a device and system for identifying types andamounts of food consumed based on food images can include the step ofautomatically identifying boundaries between different types of food inan image that contains multiple types or portions of food. In anexample, the creation of boundaries between different types of foodand/or segmentation of a meal into different food types can include edgedetection, shading analysis, texture analysis, and three-dimensionalmodeling. In an example, this process can also be informed by commonpatterns ofjointly-served foods and common boundary characteristics ofsuch jointly-served foods.

In an example, estimation of specific ingredients or nutrients consumedfrom information concerning food consumed can be done using a databasethat links specific foods (and quantities thereof) with specificingredients or nutrients (and quantities thereof). In an example, foodin a picture can be classified and identified based on comparison withpictures of known foods in a food image database. In an example, suchfood identification can be assisted by pattern recognition software. Inan example, types and quantities of specific ingredients or nutrientscan be estimated from the types and quantities of food consumed.

In an example, attributes of food in an image can be represented by amulti-dimensional food attribute vector. In an example, this foodattribute vector can be statistically compared to the attribute vectorof known foods in order to automate food identification. In an example,multivariate analysis can be done to identify the most likelyidentification category for a particular portion of food in an image. Invarious examples, a multi-dimensional food attribute vector can includeattributes selected from the group consisting of: food color; foodtexture; food shape; food size or scale; geographic location ofselection, purchase, or consumption; timing of day, week, or specialevent; common food combinations or pairings; image brightness,resolution, or lighting direction; infrared light reflection;spectroscopic analysis; and person-specific historical eating patterns.

10. Primary and Secondary Data Collection

In an example, a method for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient can comprisecollecting primary data concerning food consumption and collectingsecondary data concerning food consumption. In an example, a device andsystem for measuring a person's consumption of at least one selectedtype of food, ingredient, or nutrient can comprise a primary datacollection component and a secondary data collection component. In anexample, primary data and secondary data can be jointly analyzed toidentify the types and amounts of foods, ingredients, or nutrients thata person consumes.

In an example, primary data collection can occur automatically, withoutthe need for any specific action by a person in association with aspecific eating event, apart from the actual act of eating. In anexample, a primary data component can operate automatically, without theneed for any specific action by the person in association with aspecific eating event apart from the actual act of eating. In anexample, primary data is collected continuously, but secondary data isonly collected when primary data indicates that a person is probablyeating food. In an example, a primary data collection component operatescontinuously, but a secondary data collection component only operateswhen primary data indicates that a person is probably eating food.

In an example, primary data is collected automatically, but secondarydata is only collected when triggered, activated, or operated by aperson via a specific action in association with a specific eating eventother than the act of eating. In an example, a primary data collectioncomponent operates automatically, but a secondary data collectioncomponent only operates when it is triggered, activated, or operated bya person via a specific action in association with a specific eatingevent other than the act of eating.

In an example, collection of secondary data can require a specifictriggering or activating action by a person, apart from the act ofeating, for each specific eating event. In an example, a device tomeasure food consumption can prompt a person to trigger, activate, oroperate secondary data collection in association with a specific eatingevent when analysis of primary data indicates that this person isprobably eating. In an example, a device to measure food consumption canprompt a person to trigger, activate, or operate a secondary datacollection component in association with a specific eating event whenanalysis of primary data indicates that this person is probably eating.In an example, a component of this device that automatically collectsprimary data to detect when a person is probably eating can prompt theperson to collect secondary data to identify food consumed when theperson is probably eating. In an example, a device can prompt a personto collect secondary data in association with a specific eating eventwhen analysis of primary data indicates that the person is probablyeating and the person has not yet collected secondary data.

In an example, primary data can be collected by a wearable member andsecondary data can be collected by a hand-held member. In an example, aperson can be prompted to use a hand-held member to collect secondarydata when primary data indicates that this person is probably eating. Inan example, the wearable member can detect when a person is eatingsomething, but is not very good at identifying what selected types offood the person is eating. In an example, the hand-held member is betterat identifying what selected types of food the person is eating, butonly when the hand-held member is used, which requires specific actionby the person for each eating event.

In an example, a device and system can prompt a person to use ahand-held member (such as a mobile phone or smart utensil) to takepictures of food when a wearable member (such as a smart watch or smartbracelet) indicates that the person is probably eating. In an example, aperson can be prompted to use a digital camera to take pictures of foodwhen a wearable food-consumption monitor detects that the person isconsuming food.

In an example, a person can be prompted to use a smart utensil to takepictures of food when a wearable food-consumption monitor detects thatthe person is consuming food. In an example, a device and system canprompt a person to use a hand-held member (such as a smart utensil orfood probe) to analyze the chemical composition of food when a wearablemember (such as a smart watch or smart bracelet) indicates that theperson is probably eating. In an example, a person can be prompted touse a smart utensil for chemical analysis of food when a wearablefood-consumption monitor detects that the person is consuming food.

In an example, a device for measuring food consumption can prompt aperson to collect secondary data in real time, while a person is eating,when food consumption is indicated by primary data. In an example, adevice for measuring food consumption can prompt a person to collectsecondary data after food consumption, after food consumption has beenindicated by primary data. In various examples, a device can prompt aperson to take one or more actions to collect secondary data that areselected from the group consisting of: use a specific smart utensil forfood consumption; use a specific set of smart place setting components(dish, plate, utensils, glass, etc) to record information about typesand quantities of food; use a special food scale; touch food with a foodprobe or smart utensil; take a still picture or multiple still picturesof food from different angles; record a video of food from differentangles; and expose food to light, electromagnetic, microwave, sonic, orother energy and record the results of interaction between food and thisenergy.

In an example, the process of collecting primary data can be lessintrusive than the process of collecting secondary data with respect toa person's privacy. In an example, secondary data can enable moreaccurate food identification than primary data with respect to measuringa person's consumption of at least one selected type of food,ingredient, or nutrient. In an example, a coordinated system of primaryand secondary data collection can achieve a greater level of measurementaccuracy for a selected level of privacy intrusion than either primarydata collection or secondary data collection alone. In an example, acoordinated system of primary and secondary data collection can achievea lower level of privacy intrusion for a selected level of measurementaccuracy than either primary data collection or secondary datacollection alone.

In an example, primary data can be collected by a device or devicecomponent that a person wears on their body or clothing. In an example,primary data can be collected by a smart watch, smart bracelet, or otherwrist-worn member. In an example, primary data can be collected by asmart necklace or other neck-worn member. In an example, primary datacan be collected by smart glasses or other electronically-functionaleyewear. In an example, primary data can be data concerning a person'smovements that is collected using a motion detector. In an example, aprimary data collection component can monitor a person's movements formovements that indicate that the person is probably eating food. In anexample, primary data can be data concerning electromagnetic signalsfrom a person's body. In an example, a primary data collection componentcan monitor electromagnetic signals from the person's body for signalsthat indicate that the person is probably eating food.

In an example, secondary data can be collected by a device or devicecomponent that a person holds in their hand. In an example, secondarydata can be collected by a smart phone, mobile phone, smart utensil, orsmart food probe. In an example, secondary data can be images of food.In an example, collection of secondary data can require that the personaim a camera at food and take one or more pictures of food. In anexample, a camera-based food-identifying sensor automatically startstaking pictures when data collected by the monitor indicates that aperson is probably consuming food, but the person is prompted tomanually aim the camera toward food being consumed when data collectedby the monitor indicates that a person is probably consuming food.

In an example, secondary data can be the results of chemical analysis offood. In an example, collection of secondary data can require that theperson bring a nutrient-identifying utensil or sensor into physicalcontact with food. In an example, collection of secondary data canrequire that the person speak into a voice-recognizing device andverbally identify the food that they are eating. In an example,collection of secondary data can require that the person use acomputerized menu-interface to identify the food that they are eating.

In an example, a device for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient can collect primarydata concerning food consumption without the need for a specific actionby the person in association with an eating event apart from the act ofeating. In an example, a device for measuring a person's consumption ofat least one selected type of food, ingredient, or nutrient can collectprimary data automatically. In an example, a device for measuring aperson's consumption of at least one selected type of food, ingredient,or nutrient can collect primary data continually.

In an example, a device for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient automaticallycollects secondary data concerning food consumption during a specificeating event, but only when analysis of primary data indicates that theperson is eating. In an example, a device for measuring a person'sconsumption of at least one selected type of food, ingredient, ornutrient only collects secondary data concerning food consumption duringa specific eating when it is triggered, activated, or operated by theperson for that eating event by an action apart from the act of eating.In an example, a device can prompt the person to trigger, activate, oroperate secondary data collection when primary data indicates that theperson is eating.

In an example, a device for measuring a person's food consumption canautomatically start collecting secondary data when primary data detects:reachable food sources; hand-to-food interaction; physical location in arestaurant, kitchen, dining room, or other location associated withprobable food consumption; hand or arm motions associated with bringingfood up to the person's mouth; physiologic responses by the person'sbody that are associated with probable food consumption; smells orsounds that are associated with probable food consumption; and/or speechpatterns that are associated with probable food consumption.

In an example, a device for measuring a person's food consumption canprompt a person to collect secondary data when primary data detects:reachable food sources; hand-to-food interaction; physical location in arestaurant, kitchen, dining room, or other location associated withprobable food consumption; hand or arm motions associated with bringingfood up to the person's mouth; physiologic responses by the person'sbody that are associated with probable food consumption; smells orsounds that are associated with probable food consumption; and/or speechpatterns that are associated with probable food consumption.

In an example, a device for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient can include acombination of food identification methods or steps that are performedautomatically by a computer and food identification methods or stepsthat are performed by a human. In an example, a device and method fordetecting food consumption and identifying consumption of specificingredients or nutrients can comprise multiple types of data collectionand analysis involving interaction between automated analysis and humanentry of information. In an example, a person can play a role insegmenting an image of a multi-food meal into different types of food bycreating a virtual boundary between foods, such as by moving theirfinger across a touch-screen image of the meal. In an example, theperson may review images of food consumed after an eating event andmanually enter food identification information. In an example, a personcan select one or more food types and/or quantities from a menu providedin response to a picture or other recorded evidence of an eating event.

In an example, redundant food identification can be performed by both acomputer and a human during a calibration period, after which foodidentification is performed only by a computer. In an example, a deviceand system can automatically calibrate sensors and responses based onknown quantities and outcomes. In an example, a person can eat food withknown amounts of specific ingredients or nutrients. In an example,measured amounts can be compared to known amounts in order to calibratedevice or system sensors. In an example, a device and system can trackactual changes in a person's weight or Body Mass Index (BMI) and usethese actual changes to calibrate device or system sensors. In anexample, a device or system for measuring a person's consumption of atleast one specific food, ingredient, or nutrient can be capable ofadaptive machine learning. In an example, such a device or system caninclude a neural network. In an example, such a device and system caniteratively adjust the weights given to human responses based onfeedback and health outcomes

In an example, initial estimates of the types and amounts of foodconsumed can be made by a computer in an automated manner and thenrefined by human review as needed. In an example, if automated methodsfor identification of the types and amounts of food consumed do notproduce results with a required level of certainty, then a device andsystem can prompt a person to collect and/or otherwise providesupplemental information concerning the types of food that the person isconsuming. In an example, a device and system can track the accuracy offood consumption information provided by an automated process vs. thatprovided by a human by comparing predicted to actual changes in aperson's weight. In an example, the relative weight which a device andsystem places on information from automated processes vs. informationfrom human input can be adjusted based on their relatively accuracy inpredicting weight changes. Greater weight can be given to theinformation source which is more accurate based on empirical validation.

In an example, a device can ask a person clarifying questions concerningfood consumed. In an example, a device can prompt the person withqueries to refine initial automatically-generated estimates of the typesand quantities of food consumed. In an example, these questions can beasked in real time, as a person is eating, or in a delayed manner, aftera person has finished eating or at a particular time of the day. In anexample, the results of preliminary automated food identification can bepresented to a human via a graphical user interface and the human canthen refine the results using a touch screen. In an example, the resultsof automated food identification can be presented to a human via verbalmessage and the human can refine the results using a speech recognitioninterface. In an example, data can be transmitted (such as by theinternet) to a review center where food is identified by a dietician orother specialist. In various examples, a human-to-computer interface forentering information concerning food consumption can comprise one ormore interface elements selected the group consisting of: microphone,speech recognition, and/or voice recognition interface; touch screen,touch pad, keypad, keyboard, buttons, or other touch-based interface;camera, motion recognition, gesture recognition, eye motion tracking, orother motion detection interface; interactive food-identification menuwith food pictures and names; and interactive food-identification searchbox.

In an example, a device and method for measuring consumption of aselected type of food, ingredient, or nutrient can comprise: a wearablemotion sensor that is worn by a person that automatically collects dataconcerning the person's body motion, wherein this body motion data isused to determine when this person is consuming food; and a userinterface that prompts the person to provide additional informationconcerning the selected types of foods, ingredients, or nutrients thatthe person is eating when the body motion data indicates that the personis consuming food.

In an example, a device and method for measuring consumption of aselected type of food, ingredient, or nutrient can comprise: a wearablesound sensor that is worn by a person that automatically collects dataconcerning sounds from the person's body or the environment, whereinthis sound data is used to determine when this person is consuming food;and a user interface that prompts the person to provide additionalinformation concerning the selected types of foods, ingredients, ornutrients that the person is eating when the sound data indicates thatthe person is consuming food.

In an example, a device and method for measuring consumption of aselected type of food, ingredient, or nutrient can comprise: a wearableimaging sensor that is worn by a person that automatically collectsimage data, wherein this image data is used to determine when thisperson is consuming food; and a user interface that prompts the personto provide additional information concerning the selected types offoods, ingredients, or nutrients that the person is eating when theimaging data indicates that the person is consuming food.

In an example, a device for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient can comprise awearable camera that continually takes pictures of the space surroundinga person. In an example, a camera can continually track the locations ofa person's hands and only focus on the space near those hands to detectpossible hand-and-food interaction. In an example, a device formonitoring a person's food consumption can optically monitor the spacearound a person for reachable food sources that may result in foodconsumption. In an example, a device for monitoring a person's foodconsumption can monitor the person's movements for hand-to-mouthgestures that may indicate food consumption.

In an example, a device can automatically recognize people within itsrange of vision and restrict picture focal range or content to notrecord pictures of people. In an example, this camera can automaticallydefocus images of other people for the sake of privacy. As analternative way to address privacy issues, this camera can only betriggered to take record pictures when there are visual, sonic,olfactory, or locational indicators that the person is eating food orlikely to eat food. As another way to address privacy issues, thiscamera can have a manual shut-off that the person can use to shut offthe camera.

In an example, a wearable device and system for measuring a person'sconsumption of at least one selected type of food, ingredient, ornutrient can be tamper resistant. In an example, a wearable device candetect when it has been removed from the person's body by monitoringsignals from the body such as pulse, motion, heat, skinelectromagnetism, or proximity to an implanted device. In an example, awearable device for measuring food consumption can detect if it has beenremoved from the person's body by detecting a lack of motion, lack of apulse, and/or lack of electromagnetic response from skin. In variousexamples, a wearable device for measuring food consumption cancontinually monitor optical, electromagnetic, temperature, pressure, ormotion signals that indicate that the device is properly worn by aperson. In an example, a wearable device can trigger feedback if thedevice is removed from the person and the signals stop.

In an example, a wearable device for measuring food consumption candetect if its mode of operation becomes impaired. In an example, awearable device for measuring food consumption that relies on takingpictures of food can detect if its line-of-sight to a person's hands ormouth is blocked. In an example, a wearable device can automaticallytrack the location of a person's hands or mouth and can trigger feedbackif this tracking is impaired. In an example, wrist-worn devices can beworn on both wrists to make monitoring food consumption more inclusiveand to make it more difficult for a person to circumvent detection offood consumption by the combined devices or system. In an example, awearable device for measuring food consumption that relies on a smartfood utensil can detect if a person is consuming food without using thesmart utensil. In an example, a device or system can detect when autensil or food probe is not in functional linkage with wearable member.In an example, functional linkage can be monitored by common movement,common sound patterns, or physical proximity. In an example, a device orsystem can trigger feedback or behavioral modification if its functionis impaired.

In an example, a person can be prompted to use a hand-heldfood-identifying sensor to identify the type of food being consumed whena smart watch detects that the person is consuming food and thehand-held food-identifying sensor is not already being used. In anexample, a device and system for monitoring, sensing, detecting, and/ortracking a person's consumption of one or more selected types of foods,ingredients, or nutrients can comprise a wearable food-consumptionmonitor (such as a smart watch or smart necklace) and a hand-heldfood-identifying sensor (such as a smart utensil or smart phone),wherein data collected by the monitor and sensor are jointly analyzed tomeasure the types and amounts of specific foods, ingredients, and/ornutrients that the person consumes.

In an example, a person can be prompted to use a hand-heldfood-identifying sensor for chemical analysis of food when a smart watchdetects that the person is consuming food. In an example, a person canbe prompted to use a smart utensil for chemical analysis of food when asmart watch detects that the person is consuming food. In an example, aperson can be prompted to use a food probe for chemical analysis of foodwhen a smart watch detects that the person is consuming food.

In an example, a person can be prompted to use a hand-heldfood-identifying sensor to take pictures of food when a smart watchdetects that the person is consuming food. In an example, a person canbe prompted to use a mobile phone to take pictures of food when a smartwatch detects that the person is consuming food. In an example, a personcan be prompted to use a smart utensil to take pictures of food when asmart watch detects that the person is consuming food. In an example, aperson can be prompted to use a digital camera to take pictures of foodwhen a smart watch detects that the person is consuming food.

In an example, a device and method for monitoring, sensing, detecting,and/or tracking a person's consumption of one or more selected types offoods, ingredients, or nutrients can comprise a wearable device withprimary and second modes, mechanisms, or levels of data collectionconcerning a person's food consumption. The primary mode of datacollection can be continuous, not requiring action by the person inassociation with an eating event apart from the act of eating, and bemore useful for general detection of food consumption than it is foridentification of consumption of selected types of foods, ingredients,and/or nutrients by the person. The secondary mode of data collectioncan be non-continuous, requiring action by the person in associationwith an eating event apart from the act of eating, and can be veryuseful for identification of consumption of selected types of foods,ingredients, and/or nutrients by the person.

In an example, both primary and secondary data collection can beperformed by a device that a person wears on their wrist (such as asmart watch or watch phone). In example, both primary and secondary datacollection can be performed by a device that a person wears around theirneck (such as a smart necklace or necklace phone). In an example,primary and secondary data can be jointly analyzed to measure the typesand amounts of specific foods, ingredients, and/or nutrients that theperson consumes. In an example, a person can be prompted to collectsecondary data when primary data indicates that the person is probablyconsuming food.

In an example, data collection by a hand-held food-identifying sensor(such as a smart utensil, food probe, or smart phone) concerning aparticular eating event requires action by a person in association withthis eating event apart from the actual act of eating. In an example,the person can be prompted to collect data using the hand-heldfood-identifying sensor when: data that is automatically collected by awearable food-consumption monitor indicates that the person is probablyconsuming food; and the person has not already collected data concerningthis particular eating event.

In an example, data collection by a hand-held food-identifying sensorcan require that a person bring a food-identifying sensor into contactwith food, wherein the person is prompted to bring the food-identifyingsensor into contact with food when: data that is automatically collectedby a wearable food-consumption monitor indicates that the person isprobably consuming food; and the person has not already brought thefood-identifying sensor into contact with this food. In an example, datacollection by a hand-held food-identifying sensor can require that theperson aim a camera and take a picture of food, wherein the person isprompted to aim a camera and take a picture of food when: data that isautomatically collected by a wearable food-consumption monitor indicatesthat the person is probably consuming food; and the person has notalready taken a picture of this food.

In an example, data collection by a hand-held food-identifying sensorcan require that a person enter information concerning food consumedinto a hand-held member by touch, keyboard, speech, or gesture. Theperson can be prompted to enter information concerning food consumedinto a hand-held member by touch, keyboard, speech, or gesture when:data that is automatically collected by a wearable food-consumptionmonitor indicates that the person is probably consuming food; and theperson has not already entered information concerning this food.

11. Some Devices and Methods for Measuring Food Consumption

In an example, a device for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient can comprise: awearable food-consumption monitor that detects when the person isprobably consuming food; and a hand-held food-identifying sensor thatdetects the person's consumption of at least one selected type of food,ingredient, or nutrient. In an example, the person can be prompted touse the hand-held food-identifying sensor when the wearable consumptionmonitor indicates that the person is consuming food. In an example, thehand-held food-identifying sensor can be automatically activated ortriggered when the food-consumption monitor indicates that the person isconsuming food.

In an example, a device for measuring, monitoring, sensing, detecting,and/or tracking a person's consumption of at least one selected type offood, ingredient, or nutrient can comprise: a wearable food-consumptionmonitor that automatically monitors and detects when the person consumesfood, wherein operation of this monitor to detect food consumption doesnot require any action associated with a particular eating event by theperson apart from the actual act of eating; and a hand-heldfood-identifying sensor that identifies the selected types of foods,ingredients, and/or nutrients that the person consumes, whereinoperation of this sensor to identify foods, ingredients, and/ornutrients during a particular eating event requires action by the personapart associated with that eating event apart from the actual act ofeating, and wherein the person is prompted to use the hand-heldfood-identifying sensor when the wearable consumption monitor indicatesthat the person is consuming food.

In an example, a method for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient can comprise:collecting primary data concerning food consumption using a wearablefood-consumption monitor to detect when a person is consuming food; andcollecting secondary data concerning food consumption using a hand-heldfood-identifying sensor when analysis of primary data indicates that theperson is consuming food. In an example, collection of secondary datacan be automatic when primary data indicates that the person isconsuming food. In an example, collection of secondary data can requirea triggering action by the person in association with a particulareating event apart from the actual act of eating. In an example, theperson can be prompted to take the triggering action necessary tocollect secondary data when primary data indicates that the person isconsuming food.

In an example, a method for measuring, monitoring, sensing, detecting,and/or tracking a person's consumption of at least one selected type offood, ingredient, or nutrient can comprise: collecting primary datausing a wearable food-consumption monitor to detect when a person isprobably consuming food, wherein this detector is worn on the person,and wherein primary data collection does not require action by theperson at the time of food consumption apart from the act of consumingfood; and collecting secondary data using a hand-held food-identifyingsensor to identify the selected types of foods, ingredients, ornutrients that the person is consuming, wherein secondary datacollection by the hand-held food-identifying sensor requires action bythe person at the time of food consumption apart from the act ofconsuming food, and wherein the person is prompted to take this actionwhen primary data indicates that the person is consuming food andsecondary data has not already been collected.

In an example, a method for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient can comprise: (a)having the person wear a motion sensor that is configured to be worn onat least one body member selected from the group consisting of wrist,hand, finger, and arm; wherein this motion sensor continually monitorsbody motion to provide primary data that is used to detect when a personis consuming food; (b) prompting the person to collect secondary dataconcerning food consumption when this primary data indicates that theperson is consuming food; wherein secondary data is selected from thegroup consisting of: data from the interaction between food andreflected, absorbed, or emitted light energy including pictures,chromatographic results, fluorescence results, absorption spectra,reflection spectra, infrared radiation, and ultraviolet radiation; datafrom the interaction between food and electromagnetic energy includingelectrical conductivity, electrical resistance, and magneticinteraction; data from the interaction between food and sonic energyincluding ultrasonic energy; data from the interaction between food andchemical receptors including reagents, enzymes, biological cells, andmicroorganisms; and data from the interaction between food and massmeasuring devices including scales and inertial sensors; and (c) usingboth primary and secondary data to identify the types and quantities offood consumed in a manner that is at least a partially-automatic;wherein the identification of food type and quantity includes one ormore methods selected from the group consisting of: motion patternanalysis and identification; image pattern analysis and identification;chromatography; electromagnetic energy pattern analysis andidentification; sound pattern analysis and identification; mass, weight,and/or density; and chemical composition analysis.

In an example, a device for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient can comprise: awearable motion sensor that automatically collects data concerning bodymotion, wherein this body motion data is used to determine when a personis consuming food; and an imaging sensor that collects images of food,wherein these food images are used to identify the type and quantity offood, ingredients, or nutrients that a person is consuming food. In anexample, an imaging sensor that requires action by the person topictures of food during an eating event. In an example, the device canprompt the person to use the imaging sensor to take pictures of foodwhen body motion data indicates that the person is consuming food. In anexample, a device for measuring a person's consumption of at least oneselected type of food, ingredient, or nutrient can comprise: a wearablemotion sensor that is worn by a person, wherein this motion sensorautomatically and continuously collects data concerning the person'sbody motion, and wherein the body motion data is used to determine whena person is consuming food; and a wearable imaging sensor that is wornby the person, wherein this imaging sensor does not continuously takepictures, but rather only collects images of eating activity when bodymotion data indicates that the person is consuming food.

In an example, an imaging sensor need not collect images continuously,but rather requires specific action by the person to initiate imaging atthe time of food consumption apart from the actual action of eating. Inan example, a person can be prompted to take pictures of food when bodymotion data collected by a wearable motion sensor indicates that theperson is consuming food. In an example, a person can be prompted totake pictures of food when sound data collected by a wearable soundsensor indicates that the person is consuming food.

In an example, a device for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient can comprise: awearable motion sensor that automatically collects data concerning bodymotion, wherein this body motion data is used to determine when a personis consuming food; and a chemical composition sensor that analyzes thechemical composition of food, wherein results of this chemical analysisare used to identify the type and quantity of food, ingredients, ornutrients that a person is consuming food. In an example, a device formeasuring a person's consumption of at least one selected type of food,ingredient, or nutrient can comprise: a wearable motion sensor that isworn by a person, wherein this motion sensor automatically andcontinuously collects data concerning the person's body motion, andwherein the body motion data is used to determine when a person isconsuming food; and a chemical composition sensor, wherein this chemicalcomposition sensor does not continuously monitor the chemicalcomposition of material within the person's mouth or gastrointestinaltract, but rather only collects information concerning the chemicalcomposition of material within the person's mouth or gastrointestinaltract when body motion data indicates that the person is consuming food.

In an example, a chemical composition sensor can identify the type offood, ingredient, or nutrient based on: physical contact between thesensor and food; or the effects of interaction between food andelectromagnetic energy or light energy. In an example, a chemicalcomposition sensor need not collect chemical information continuously,but rather requires specific action by the person to initiate chemicalanalysis at the time of food consumption apart from the actual action ofconsuming food. In an example, a person can be prompted to activate asensor to perform chemical analysis of food when body motion datacollected by a wearable motion sensor indicates that the person isconsuming food. In an example, a person can be prompted to activate asensor to perform chemical analysis of food when sound data collected bya wearable sound sensor indicates that the person is consuming food.

In an example, a device for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient can comprise: awearable sound sensor that automatically collects data concerning bodyor environmental sounds, wherein this sound data is used to determinewhen a person is consuming food; and an imaging sensor that collectsimages of food, wherein these food images are used to identify the typeand quantity of food, ingredients, or nutrients that a person isconsuming food. In an example, this imaging sensor can require action bythe person to pictures of food during an eating event. In an example,the person can be prompted to use the imaging sensor to take pictures offood when sound data indicates that the person is consuming food. In anexample, a device for measuring a person's consumption of at least oneselected type of food, ingredient, or nutrient can comprise: a wearablesound sensor that is worn by a person, wherein this sound sensorautomatically and continuously collects data concerning sounds from theperson's body, and wherein this sound data is used to determine when aperson is consuming food; and a wearable imaging sensor that is worn bythe person, wherein this imaging sensor does not continuously takepictures, but rather only collects images of eating activity when sounddata indicates that the person is consuming food.

In an example, a device for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient can comprise: awearable sound sensor that automatically collects data concerning bodyor environmental sound, wherein this sound data is used to determinewhen a person is consuming food; and a chemical composition sensor thatanalyzes the chemical composition of food, wherein results of thischemical analysis are used to identify the type and quantity of food,ingredients, or nutrients that a person is consuming food. In anexample, a device for measuring a person's consumption of at least oneselected type of food, ingredient, or nutrient can comprise: a wearablesound sensor that is worn by a person, wherein this motion sensorautomatically and continuously collects data concerning sound from theperson's body, and wherein this sound data is used to determine when aperson is consuming food; and a chemical composition sensor, whereinthis chemical composition sensor does not continuously monitor thechemical composition of material within the person's mouth orgastrointestinal tract, but rather only collects information concerningthe chemical composition of material within the person's mouth orgastrointestinal tract when sound data indicates that the person isconsuming food.

In an example, a method for measuring a person's consumption of at leastone selected type of food, ingredient, or nutrient can comprise:collecting a first set of data to detect when a person is probablyconsuming food in an automatic and continuous manner that does notrequire action by the person at the time of food consumption apart fromthe act of consuming food; collecting a second set of data to identifywhat selected types of foods, ingredients, or nutrients a person isconsuming when the first set of data indicates that the person isprobably consuming food; and jointly analyzing both the first and secondsets of data to estimate consumption of at least one specific food,ingredient, or nutrient by the person.

In an example, a device or system for measuring a person's consumptionof at least one selected type of food, ingredient, or nutrient cancomprise: (a) a wearable food-consumption monitor that is configured tobe worn on a person's body or clothing, wherein this monitorautomatically collects primary data that is used to detect when theperson is consuming food; (b) a food-identifying sensor that collectssecondary data that is used to measure the person's consumption of atleast one selected type of food, ingredient, or nutrient, and whereinsecondary data collection in association with a specific foodconsumption event requires a specific action by the person inassociation with that specific food consumption event apart from the actof consuming food; and (c) a computer-to-human prompting interface,wherein this interface prompts the person to take the specific actionrequired for secondary data collection in association with a specificfood consumption event when the primary data indicates that the personis consuming food and the person has not already taken this specificaction. In an example, primary data can be body movement data or dataconcerning electromagnetic signals from the person's body. In anexample, secondary data can be collected by a mobile phone, smartutensil, food probe, smart necklace, smart eyewear, or a smart watch.

In an example, a device or system for measuring a person's consumptionof at least one selected type of food, ingredient, or nutrient cancomprise: (a) a wearable food-consumption monitor that is configured tobe worn on a person's body or clothing, wherein this monitorautomatically collects primary data that is used to detect when theperson is consuming food; (b) an imaging component that collectssecondary data that is used to measure the person's consumption of atleast one selected type of food, ingredient, or nutrient, wherein thissecondary data comprises pictures of food, and wherein taking picturesof food in association with a specific food consumption event requires aspecific action by the person in association with that specific foodconsumption event apart from the act of consuming food; and (c) acomputer-to-human prompting interface, wherein this interface promptsthe person to take pictures of food in association with a specific foodconsumption event when the primary data indicates that the person isconsuming food and pictures of this food have not already been taken. Inan example, primary data can be body movement data or data concerningelectromagnetic signals from the person's body. In an example, secondarydata can be collected by a mobile phone, smart utensil, food probe,smart necklace, smart eyewear, or a smart watch.

In an example, a device or system for measuring a person's consumptionof at least one selected type of food, ingredient, or nutrient cancomprise: (a) a wearable food-consumption monitor that is configured tobe worn on a person's body or clothing, wherein this monitorautomatically collects primary data that is used to detect when theperson is consuming food; (b) an chemical-analyzing component thatcollects secondary data that is used to measure the person's consumptionof at least one selected type of food, ingredient, or nutrient, whereinthis secondary data comprises chemical analysis of food, and whereinperforming chemical analysis of food in association with a specific foodconsumption event requires a specific action by the person inassociation with that specific food consumption event apart from the actof consuming food; and (c) a computer-to-human prompting interface,wherein this interface prompts the person to take the action required toperform chemical analysis of food in association with a specific foodconsumption event when the primary data indicates that the person isconsuming food and chemical analysis of this food has not already beenperformed. In an example, primary data can be body movement data or dataconcerning electromagnetic signals from the person's body. In anexample, secondary data can be collected by a mobile phone, smartutensil, food probe, smart necklace, smart eyewear, or a smart watch.

In an example, a device or system for measuring a person's consumptionof at least one selected type of food, ingredient, or nutrient cancomprise: (a) a wearable food-consumption monitor that is configured tobe worn on a person's body or clothing, wherein this monitorautomatically collects primary data that is used to detect when theperson is consuming food; (b) a computer-to-human prompting interfacewhich a person uses to enter secondary data concerning the person'sconsumption of at least one selected type of food, ingredient, ornutrient, wherein this interface selected from the group consisting of:speech or voice recognition, touch or gesture recognition, motionrecognition or eye tracking, and buttons or keys, and wherein thisinterface prompts the person to enter secondary data in association witha specific food consumption event when the primary data indicates thatthe person is consuming food and the person has not already entered thisdata. In an example, primary data can be body movement data or dataconcerning electromagnetic signals from the person's body. In anexample, secondary data can be collected by a mobile phone, smartutensil, food probe, smart necklace, smart eyewear, or a smart watch.

In an example, a device or system for measuring a person's consumptionof at least one selected type of food, ingredient, or nutrient cancomprise: (a) a wearable food-consumption monitor that is configured tobe worn on a person's body or clothing, wherein this monitorautomatically collects primary data that is used to detect when theperson is consuming food; (b) a food-identifying sensor thatautomatically collects secondary data that is used to measure theperson's consumption of at least one selected type of food, ingredient,or nutrient in association with a specific food consumption event whenthe primary data indicates that the person is consuming food. In anexample, primary data can be body movement data or data concerningelectromagnetic signals from the person's body. In an example, secondarydata can be collected by a mobile phone, smart utensil, food probe,smart necklace, smart eyewear, or a smart watch.

In an example, a device or system for measuring a person's consumptionof at least one selected type of food, ingredient, or nutrient cancomprise: (a) a smart watch that is configured to be worn on a person'swrist, hand, or arm, wherein this smart watch automatically collectsprimary data that is used to detect when the person is consuming food;(b) a food-identifying sensor that collects secondary data that is usedto measure the person's consumption of at least one selected type offood, ingredient, or nutrient, and wherein secondary data collection inassociation with a specific food consumption event requires a specificaction by the person in association with that specific food consumptionevent apart from the act of consuming food; and (c) a computer-to-humanprompting interface, wherein this interface prompts the person to takethe specific action required for secondary data collection inassociation with a specific food consumption event when the primary dataindicates that the person is consuming food and the person has notalready taken this specific action. In an example, primary data can bebody movement data or data concerning electromagnetic signals from theperson's body. In an example, secondary data can be collected by amobile phone, smart utensil, food probe, smart necklace, smart eyewear,or the smart watch.

In an example, a device or system for measuring a person's consumptionof at least one selected type of food, ingredient, or nutrient cancomprise: (a) a smart watch that is configured to be worn on a person'swrist, hand, or arm, wherein this smart watch automatically collectsprimary data that is used to detect when the person is consuming food;(b) an imaging component that collects secondary data that is used tomeasure the person's consumption of at least one selected type of food,ingredient, or nutrient, wherein this secondary data comprises picturesof food, and wherein taking pictures of food in association with aspecific food consumption event requires a specific action by the personin association with that specific food consumption event apart from theact of consuming food; and (c) a computer-to-human prompting interface,wherein this interface prompts the person to take pictures of food inassociation with a specific food consumption event when the primary dataindicates that the person is consuming food and pictures of this foodhave not already been taken. In an example, primary data can be bodymovement data or data concerning electromagnetic signals from theperson's body. In an example, secondary data can be collected by amobile phone, smart utensil, food probe, smart necklace, smart eyewear,or the smart watch.

In an example, a device or system for measuring a person's consumptionof at least one selected type of food, ingredient, or nutrient cancomprise: (a) a smart watch that is configured to be worn on a person'swrist, hand, or arm, wherein this smart watch automatically collectsprimary data that is used to detect when the person is consuming food;(b) an chemical-analyzing component that collects secondary data that isused to measure the person's consumption of at least one selected typeof food, ingredient, or nutrient, wherein this secondary data compriseschemical analysis of food, and wherein performing chemical analysis offood in association with a specific food consumption event requires aspecific action by the person in association with that specific foodconsumption event apart from the act of consuming food; and (c) acomputer-to-human prompting interface, wherein this interface promptsthe person to take the action required to perform chemical analysis offood in association with a specific food consumption event when theprimary data indicates that the person is consuming food and chemicalanalysis of this food has not already been performed. In an example,primary data can be body movement data or data concerningelectromagnetic signals from the person's body. In an example, secondarydata can be collected by a mobile phone, smart utensil, food probe,smart necklace, smart eyewear, or the smart watch.

In an example, a device or system for measuring a person's consumptionof at least one selected type of food, ingredient, or nutrient cancomprise: (a) a smart watch that is configured to be worn on a person'swrist, hand, or arm, wherein this smart watch automatically collectsprimary data that is used to detect when the person is consuming food;(b) a computer-to-human prompting interface which a person uses to entersecondary data concerning the person's consumption of at least oneselected type of food, ingredient, or nutrient, wherein this interfaceselected from the group consisting of: speech or voice recognition,touch or gesture recognition, motion recognition or eye tracking, andbuttons or keys, and wherein this interface prompts the person to entersecondary data in association with a specific food consumption eventwhen the primary data indicates that the person is consuming food andthe person has not already entered this data. In an example, primarydata can be body movement data or data concerning electromagneticsignals from the person's body. In an example, the interface cancomprise a mobile phone, smart utensil, food probe, smart necklace,smart eyewear, or the smart watch.

In an example, a device or system for measuring a person's consumptionof at least one selected type of food, ingredient, or nutrient cancomprise: (a) a smart watch that is configured to be worn on a person'swrist, hand, or arm, wherein this smart watch automatically collectsprimary data that is used to detect when the person is consuming food;(b) a food-identifying sensor that automatically collects secondary datathat is used to measure the person's consumption of at least oneselected type of food, ingredient, or nutrient in association with aspecific food consumption event when the primary data indicates that theperson is consuming food. In an example, primary data can be bodymovement data or data concerning electromagnetic signals from theperson's body. In an example, secondary data can be collected by amobile phone, smart utensil, food probe, smart necklace, smart eyewear,or the smart watch.

12. Narrative to Accompany FIGS. 1 Through 4

First we will provide an introductory overview to FIGS. 1 through 4.FIGS. 1 through 4 show an example of how this invention can be embodiedin a device and system for measuring a person's consumption of at leastone specific type of food, ingredient, or nutrient, wherein this deviceand system has two components. The first component is a wearablefood-consumption monitor that is worn on a person's body or clothing. Inthis example, the wearable food-consumption monitor is a smart watchthat is worn on a person's wrist. The smart watch automatically collectsprimary data that is used to detect when a person is consuming food. Thesecond component is a hand-held food-identifying sensor. In thisexample, the hand-held food-identifying sensor is a smart spoon. Thesmart spoon collects secondary data that is used to identify theperson's consumption of at least one specific type of food, ingredient,or nutrient.

In the example shown in FIGS. 1 through 4, the smart watch collectsprimary data automatically, without requiring any specific action by theperson in association with a specific eating event apart from the actualact of eating. As long as the person continues to wear the smart watch,the smart watch collects the primary data that is used to detect foodconsumption. In an example, primary data can be motion data concerningthe person's wrist movements. In an example, primary data can beup-and-down and tilting movements of the wrist that are generallyassociated with eating food. In contrast to primary data collection bythe smart watch, which is automatic and relatively-continuous, secondarydata collection by the smart spoon depends on the person using thatparticular spoon to eat. In other words, secondary data collection bythe smart spoon requires specific action by the person in associationwith a specific eating event apart from the actual act of eating.

This device and system includes both a smart watch and a smart spoonthat work together as an integrated system. Having the smart watch andsmart spoon work together provides advantages over use of either a smartwatch or a smart spoon by itself. The smart watch provides superiorcapability for food consumption monitoring (as compared to a smartspoon) because the person wears the smart watch all the time and thesmart watch monitors for food consumption continually. The smart spoonprovides superior capability for food identification (as compared to asmart watch) because the spoon has direct contact with the food and candirectly analyze the chemical composition of food in a manner that isdifficult to do with a wrist-worn member. Having both the smart watchand smart spoon work together as an integrated system can provide bettermonitoring compliance and more-accurate food identification than eitherworking alone.

As FIGS. 1 through 4 collectively show, an integrated device and systemthat comprises both a smart watch and a smart spoon, working together,can measure a person's consumption of at least one selected type offood, ingredient, or nutrient in a more consistent and accurate mannerthan either a smart watch or a smart spoon operating alone. One way inwhich the smart watch and smart spoon can work together is for the smartwatch to track whether or not the smart spoon is being used when thesmart watch detects that the person is eating food. If the smart spoonis not being used when the person eats, then the smart watch can promptthe person to use the smart spoon. This prompt can range from arelatively-innocuous tone or vibration (which the person can easilyignore) to a more-substantive aversive stimulus, depending on thestrength of the person's desire for measurement accuracy andself-control.

Having provided an introductory overview for FIGS. 1 through 4collectively, we now discuss them individually. FIG. 1 introduces thehand-held food-identifying sensor of this device, which is a smart spoonin this example. In this example, a smart spoon is a specializedelectronic spoon that includes food sensors as well as wireless datacommunication capability. In this example, the smart spoon includes achemical sensor which analyzes the chemical composition of food withwhich the spoon comes into contact. FIG. 2 introduces the wearablefood-consumption monitor of this device, which is a smart watch in thisexample. In this example, a smart watch is a wrist-worn electronicdevice that includes body sensors, a data processing unit, and wirelessdata communication capability. In this example, the body sensor is amotion sensor. FIGS. 3 and 4 show how the smart spoon and smart watchwork together as an integrated system to monitor and measure a person'sconsumption of at least one selected type of food, ingredient, ornutrient. We now discuss FIGS. 1 through 4 individually in more detail.

FIG. 1 shows that the hand-held food-identifying sensor in this deviceis a smart spoon 101 that comprises at least four operationalcomponents: a chemical composition sensor 102; a data processing unit103; a communication unit 104; and a power supply and/or transducer 105.In other examples, the hand-held food-identifying sensor component ofthis device can be a different kind of smart utensil, such as a smartfork, or can be a hand-held food probe. In an example, smart spoon 101can include other components, such as a motion sensor or camera. Thefour operational components 102-105 of smart spoon 101 in this exampleare in electronic communication with each other. In an example, thiselectronic communication can be wireless. In another example, thiselectronic communication can be through wires. Connecting electroniccomponents with wires is well-known in the prior art and the preciseconfiguration of possible wires is not central to this invention, soconnecting wires are not shown.

In an example, power supply and/or transducer 105 can be selected fromthe group consisting of: power from a power source that is internal tothe device during regular operation (such as an internal battery,capacitor, energy-storing microchip, or wound coil or spring); powerthat is obtained, harvested, or transduced from a power source otherthan the person's body that is external to the device (such as arechargeable battery, electromagnetic inductance from external source,solar energy, indoor lighting energy, wired connection to an externalpower source, ambient or localized radiofrequency energy, or ambientthermal energy); and power that is obtained, harvested, or transducedfrom the person's body (such as kinetic or mechanical energy from bodymotion.

In the example shown in FIG. 1, chemical composition sensor 102 on thefood-carrying scoop end of smart spoon 101 can identify at least oneselected type of food, ingredient, or nutrient by analyzing the chemicalcomposition of food that is carried by smart spoon 101. In this example,chemical composition sensor 102 analyzes the chemical composition offood by being in direct fluid communication with food that is carried inthe scoop end of smart spoon 101. In this example, chemical compositionsensor 102 includes at least one chemical receptor to which chemicals ina selected type of food, ingredient, or nutrient bind. This bindingaction creates a signal that is detected by the chemical compositionsensor 102, received by the data processing unit 103, and thentransmitted to a smart watch or other location via communication unit104.

In another example, chemical composition sensor 102 can analyze thechemical composition of food by measuring the effects of the interactionbetween food and light energy. In an example, this interaction cancomprise the degree of reflection or absorption of light by food atdifferent light wavelengths. In an example, this interaction can includespectroscopic analysis.

In an example, chemical composition sensor 102 can directly identify atleast one selected type of food by chemical analysis of food contactedby the spoon. In an example, chemical composition sensor 102 candirectly identify at least one selected type of ingredient or nutrientby chemical analysis of food. In an example, at least one selected typeof ingredient or nutrient can be indentified indirectly by: firstidentifying a type and amount of food; and then linking that identifiedfood to common types and amounts of ingredients or nutrients, using adatabase that links specific foods to specific ingredients or nutrients.In various examples, such a food database can be located in the dataprocessing unit 103 of smart spoon 101, in the data processing unit 204of a smart watch 201, or in an external device with which smart spoon101 and/or a smart watch 201 are in wireless communication.

In various examples, a selected type of food, ingredient, or nutrientthat is identified by chemical composition sensor 102 can be selectedfrom the group consisting of: a specific type of carbohydrate, a classof carbohydrates, or all carbohydrates; a specific type of sugar, aclass of sugars, or all sugars; a specific type of fat, a class of fats,or all fats; a specific type of cholesterol, a class of cholesterols, orall cholesterols; a specific type of protein, a class of proteins, orall proteins; a specific type of fiber, a class of fiber, or all fiber;a specific sodium compound, a class of sodium compounds, and all sodiumcompounds; high-carbohydrate food, high-sugar food, high-fat food, friedfood, high-cholesterol food, high-protein food, high-fiber food, andhigh-sodium food.

In various examples, chemical composition sensor 102 can analyze foodcomposition to identify one or more potential food allergens, toxins, orother substances selected from the group consisting of: ground nuts,tree nuts, dairy products, shell fish, eggs, gluten, pesticides, animalhormones, and antibiotics. In an example, a device can analyze foodcomposition to identify one or more types of food (such as pork) whoseconsumption is prohibited or discouraged for religious, moral, and/orcultural reasons.

In various examples, chemical composition sensor 102 can be selectedfrom the group of sensors consisting of: receptor-based sensor,enzyme-based sensor, reagent based sensor, antibody-based receptor,biochemical sensor, membrane sensor, pH level sensor, osmolality sensor,nucleic acid-based sensor, or DNA/RNA-based sensor; biomimetic sensor(such as an artificial taste bud or an artificial olfactory sensor),chemiresistor, chemoreceptor sensor, electrochemical sensor,electroosmotic sensor, electrophoresis sensor, or electroporationsensor; specific nutrient sensor (such as a glucose sensor, acholesterol sensor, a fat sensor, a protein-based sensor, or an aminoacid sensor); color sensor, colorimetric sensor, photochemical sensor,chemiluminescence sensor, fluorescence sensor, chromatography sensor(such as an analytical chromatography sensor, a liquid chromatographysensor, or a gas chromatography sensor), spectrometry sensor (such as amass spectrometry sensor), spectrophotometer sensor, spectral analysissensor, or spectroscopy sensor (such as a near-infrared spectroscopysensor); and laboratory-on-a-chip or microcantilever sensor.

In an example, smart spoon 101 can measure the quantities of foods,ingredients, or nutrients consumed as well as the specific types offoods, ingredients, or nutrients consumed. In an example, smart spoon101 can include a scale which tracks the individual weights (andcumulative weight) of mouthfuls of food carried and/or consumed duringan eating event. In an example, smart spoon 101 can approximate theweights of mouthfuls of food carried by the spoon by measuring theeffect of those mouthfuls on the motion of the spoon as a whole or therelative motion of one part of the spoon relative to another. In anexample, smart spoon 101 can include a motion sensor and/or inertialsensor. In an example, smart spoon 101 can include one or moreaccelerometers in different, motion-variable locations along the lengthof the spoon. In an example, smart spoon 101 can include a spring and/orstrain gauge between the food-carrying scoop of the spoon and the handleof the spoon. In an example, food weight can estimated by measuringdistension of the spring and/or strain gauge as food is brought up to aperson's mouth.

In an example, smart spoon 101 can use a motion sensor or an inertialsensor to estimate the weight of the food-carrying scoop of the spoon ata first point in time (such as during an upswing motion as the spooncarries a mouthful of food up to the person's mouth) and also at asecond point in time (such as during a downswing motion as the personlowers the spoon from their mouth). In an example, smart spoon 101 canestimate the weight of food actually consumed by calculating thedifference in food weights between the first and second points in time.In an example, a device can track cumulative food consumption bytracking the cumulative weights of multiple mouthfuls of (differenttypes of) food during an eating event or during a defined period of time(such as a day or week).

FIG. 2 shows that, in this embodiment of the invention, the wearablefood-consumption monitor component of the device is a smart watch 201.Smart watch 201 is configured to be worn around the person's wrist,adjoining the person's hand 206. In other examples, the wearablefood-consumption monitor component of this device can be embodied in asmart bracelet, smart arm band, or smart finger ring. In this example,smart watch 201 includes four operational components: a communicationunit 202; a motion sensor 203; a data processing unit 204; and a powersupply and/or transducer 205. In other examples, a wearablefood-consumption monitor component of this device can be embodied in asmart necklace. In the case of a smart necklace, monitoring for foodconsumption would more likely be done with a sound sensor rather than amotion sensor. In the case of a smart necklace, food consumption can bemonitored and detected by detecting swallowing and/or chewing sounds,rather than monitoring and detecting hand-to-mouth motions.

The four components 202-205 of smart watch 201 are in electroniccommunication with each other. In an example, this electroniccommunication can be wireless. In another example, this electroniccommunication can be through wires. Connecting electronic componentswith wires is well-known in the prior art and the precise configurationof possible wires is not central to this invention, so a configurationof connecting wires is not shown.

In an example, power supply and/or transducer 205 can be selected fromthe group consisting of: power from a power source that is internal tothe device during regular operation (such as an internal battery,capacitor, energy-storing microchip, or wound coil or spring); powerthat is obtained, harvested, or transduced from a power source otherthan the person's body that is external to the device (such as arechargeable battery, electromagnetic inductance from external source,solar energy, indoor lighting energy, wired connection to an externalpower source, ambient or localized radiofrequency energy, or ambientthermal energy); and power that is obtained, harvested, or transducedfrom the person's body (such as kinetic or mechanical energy from bodymotion.

In an example, motion sensor 203 of smart watch 201 can be selected fromthe group consisting of: bubble accelerometer, dual-axial accelerometer,electrogoniometer, gyroscope, inclinometer, inertial sensor, multi-axisaccelerometer, piezoelectric sensor, piezo-mechanical sensor, pressuresensor, proximity detector, single-axis accelerometer, strain gauge,stretch sensor, and tri-axial accelerometer. In an example, motionsensor 203 can collect primary data concerning movements of a person'swrist, hand, or arm.

In an example, there can be an identifiable pattern of movement that ishighly-associated with food consumption. Motion sensor 203 cancontinuously monitor a person's wrist movements to identify times whenthis pattern occurs to detect when the person is probably eating. In anexample, this movement can include repeated movement of the person'shand 206 up to their mouth. In an example, this movement can include acombination of three-dimensional roll, pitch, and yaw by a person'swrist. In an example, motion sensor 203 can also be used to estimate thequantity of food consumed based on the number of motion cycles. In anexample, motion sensor 203 can be also used to estimate the speed offood consumption based on the speed or frequency of motion cycles.

In various examples, movements of a person's body that can be monitoredand analyzed can be selected from the group consisting of: handmovements, wrist movements, arm movements, tilting movements, liftingmovements, hand-to-mouth movements, angles of rotation in threedimensions around the center of mass known as roll, pitch and yaw, andFourier Transformation analysis of repeated body member movements.

In various examples, smart watch 201 can include a sensor to monitor forpossible food consumption other than a motion sensor. In variousexamples, smart watch 201 can monitor for possible food consumptionusing one or more sensors selected from the group consisting of:electrogoniometer or strain gauge; optical sensor, miniature stillpicture camera, miniature video camera, miniature spectroscopy sensor;sound sensor, miniature microphone, speech recognition software, pulsesensor, ultrasound sensor; electromagnetic sensor, skin galvanicresponse (Galvanic Skin Response) sensor, EMG sensor, chewing sensor,swallowing sensor; and temperature sensor, thermometer, or infraredsensor.

In addition to smart watch 201 that is worn around the person's wrist,FIG. 2 also shows that the person's hand 206 holding a regular spoon 207that is carrying a mouthful of food 208. It is important to note thatthis is a regular spoon 207 (with no sensor or data transmissioncapability), not the smart spoon 101 that was introduced in FIG. 1.There are multiple possible reasons for use of a regular spoon 207rather than smart spoon 101. In various examples, the person may simplyhave forgotten to use the smart spoon, may be intentionally trying to“cheat” on dietary monitoring by not using the smart spoon, or may be indining setting where they are embarrassed to use the smart spoon.

In any event, if the person continues to use the regular spoon 207instead of the smart spoon 101, then the device and system will not beable to accurately identify the amounts and types of food that they areeating. If the person were not wearing smart watch 201, then the personcould continue eating with regular spoon 207 and the device would becompletely blind to the eating event. This would lead to low accuracyand low consistency in measuring food consumption. This highlights theaccuracy, consistency, and compliance problems that occur if a devicerelies only on a hand-held food-identifying sensor (without integrationwith a wearable food-consumption monitor). FIGS. 3 and 4 show how theembodiment disclosed here, comprising both a wearable food-consumptionmonitor (smart watch 201) and a hand-held food-identification sensor(smart spoon 101) that work together, can correct these problems.

In FIG. 3, motion sensor 203 of smart watch 201 detects the distinctivepattern of wrist and/or arm movement (represented symbolically by therotational dotted line arrow around hand 206) that indicates that theperson is probably consuming food. In an example, a three-dimensionalaccelerometer on smart watch 201 can detect a distinctive pattern ofupward (hand-up-to-mouth) arm movement, followed by a distinctivepattern of tilting or rolling motion (food-into-mouth) wrist movement,followed by a distinctive pattern of downward (hand-down-from-mouth)movement.

If smart watch 201 detects a distinctive pattern of body movements thatindicates that the person is probably eating and smart watch 201 has notyet received food identifying secondary data from the use of smart spoon101, then smart watch 201 can prompt the person to start using smartspoon 101. In an example, this prompt can be relatively-innocuous andeasy for the person to ignore if they wish to ignore it. In an example,this prompt can be a quiet tone, gentle vibration, or modest textmessage to a mobile phone. In another example, this prompt can be arelatively strong and aversive negative stimulus. In an example, thisprompt can be a loud sound, graphic warning, mild electric shock, and/orfinancial penalty.

In the example shown in FIG. 3, the person is not using smart spoon 101(as they should). This is detected by smart watch 201, which prompts theperson to start using smart spoon 101. In FIG. 3, this prompt 301 isrepresented by a “lightning bolt symbol”. In this example, the prompt301 represented by the lightning bolt symbol is a mild vibration. In anexample, a prompt 301 can be more substantive and/or adverse. In anexample, the prompt 301 can involve a wireless signal that to a mobilephone or other intermediary device. In an example, the prompt to theperson be communicated through an intermediary device and result in anautomated text message or phone call (through a mobile phone, forexample) to the person to prompt them to use the smart spoon.

In an example, communication unit 202 of smart watch 201 comprises acomputer-to-human interface. In an example, part of thiscomputer-to-human interface 202 can include having the computer promptthe person to collect secondary data concerning food consumption whenprimary data indicates that the person is probably consuming food. Invarious examples, communication unit 202 can use visual, auditory,tactile, electromagnetic, gustatory, and/or olfactory signals to promptthe person to use the hand-held food-identifying sensor (smart spoon 101in this example) to collect secondary data (food chemical compositiondata in this example) when primary data (motion data in this example)collected by the smart watch indicates that the person is probablyeating and the person has not already collected secondary data inassociation with a specific eating event.

In this example, the person's response to the prompt 301 from smartwatch 201 is entirely voluntary; the person can ignore the prompt andcontinue eating with a regular spoon 207 if they wish. However, if theperson wishes to have a stronger mechanism for self-control andmeasurement compliance, then the person can select (or adjust) a deviceto make the prompt stronger and less voluntary. In an example, astronger prompt can be a graphic display showing the likely impact ofexcessive food consumption, a mild electric shock, an automatic messageto a health care provider, and an automatic message to a supportivefriend or accountability partner. In an example, the prompt can compriseplaying the latest inane viral video song that is sweeping theinternet—which the person finds so annoying that they comply and switchfrom using regular spoon 207 to using smart spoon 101. The strength ofthe prompt can depend on how strongly the person feels aboutself-constraint and self-control in the context of monitoring andmodifying their patterns of food consumption.

In an example, even if a person's response to prompt 301 is entirelyvoluntary and the person ignores prompt 301 to use the smart spoon tocollect detailed secondary data concerning the meal or snack that theperson is eating, the device can still be aware that a meal or snack hasoccurred. In this respect, even if the person's response to prompt 301is voluntary, the overall device and system disclosed herein can stilltrack all eating events. This disclosed device provides greatercompliance and measurement information than is likely with a hand-helddevice only. With a hand-held device only, if the person does not usethe hand-held member for a particular eating event, then the device iscompletely oblivious to that eating event. For example, if a devicerelies on taking pictures from a smart phone to measure food consumptionand a person just keeps the phone in their pocket or purse when they eata snack or meal, then the device is oblivious to that snack or meal. Thedevice disclosed herein corrects this problem. Even if the person doesnot respond to the prompt, the device still knows that an eating eventhas occurred.

In an example, there are other ways by which smart watch 201 can detectif smart spoon 101 is being properly used or not. In an example, bothsmart watch 201 and smart spoon 101 can have integrated motion sensors(such as paired accelerometers) and their relative motions can becompared. If the movements of smart watch 201 and smart spoon 101 aresimilar during a time when smart watch 201 detects that the person isprobably consuming food, then smart spoon 101 is probably being properlyused to consume food. However, if smart spoon is not moving when smartwatch 201 detects food consumption, then smart spoon 101 is probablyjust lying somewhere unused and smart watch 201 can prompt the person touse smart spoon 101.

In a similar manner, there can be a wireless (or non-wireless physicallinkage) means of detecting physical proximity between smart watch 201and smart spoon 101. When the person is eating and the smart spoon 101is not close to smart watch 201, then smart watch 201 can prompt theperson to use smart spoon 101. In an example, physical proximity betweensmart watch 201 and smart spoon 101 can be detected by electromagneticsignals. In an example, physical proximity between smart watch 201 andsmart spoon 101 can be detected by optical signals.

If a person feels very strongly about the need for self-constraint andself-control in the measurement and modification of their foodconsumption, then a device for measuring consumption of at least oneselected type of food, ingredient, or nutrient can be madetamper-resistant. In the example shown in FIGS. 1 through 4, smart watch201 can include a mechanism for detecting when it is removed from theperson's body. This can help make it tamper-resistant. In an example,smart watch 201 can monitor signals related to the person's bodyselected from the group consisting of: pulse, motion, heat,electromagnetic signals, and proximity to an implanted device. In anexample, smart watch 201 can detect when it is been removed from theperson's wrist by detecting a lack of motion, lack of a pulse, and/orlack of electromagnetic response from skin. In various examples, smartwatch 201 can continually monitor optical, electromagnetic, temperature,pressure, or motion signals that indicate that smart watch 201 isproperly worn by a person. In an example, smart watch 201 can triggerfeedback if it is removed from the person.

In the final figure of this sequence, FIG. 4 shows that the person hasresponded positively to prompting signal 301 and has switched from usingregular spoon 207 (without food sensing and identification capability)to using smart spoon 101 (with food sensing and identificationcapability). In FIG. 4, the mouthful of food 208 that is being carriedby smart spoon 101 is now in fluid or optical communication withchemical composition sensor 102. This enables identification of at leastone selected type of food, ingredient, or nutrient by chemicalcomposition sensor 102 as part of smart spoon 101.

In an example, secondary data concerning the type of food, ingredient,or nutrient carried by smart spoon 101 can be wirelessly transmittedfrom communication unit 104 on smart spoon 101 to communication unit 202on smart watch 201. In an example, the data processing unit 204 on smartwatch 201 can track the cumulative amount consumed of at least oneselected type of food, ingredient, or nutrient. In an example, smartwatch 201 can convey this data to an external device, such as throughthe internet, for cumulative tracking and analysis.

In some respects there can be a tradeoff between the accuracy andconsistency of food consumption measurement and a person's privacy. Thedevice disclosed herein offers good accuracy and consistency of foodconsumption measurement, with relatively-low privacy intrusion. Incontrast, consider a first method of measuring food consumption that isbased only on voluntary use of a hand-held smart phone or smart utensil,apart from any wearable food consumption monitor. This first method canoffer relatively-low privacy intrusion, but the accuracy and consistencyof measurement depends completely on the person's remembering to use iteach time that the person eats a meal or snack—which can be problematic.Alternatively, consider a second method of measuring food consumptionthat is based only on a wearable device that continually records videopictures of views (or continually records sounds) around the person.This second method can offer relatively high accuracy and consistency offood consumption measurement, but can be highly intrusive with respectto the person's privacy.

The device disclosed herein provides a good solution to this problem ofaccuracy vs. privacy and is superior to either the first or secondmethods discussed above. This embodiment of this device that is shown inFIGS. 1 through 4 comprises a motion-sensing smart watch 201 and achemical-detecting smart spoon 101 that work together to offerrelatively-high food measurement accuracy with relatively-low privacyintrusion. Consistent use of the smart watch 201 does not require that aperson remember to carry, pack, or otherwise bring a particular piece ofportable electronic equipment like methods that rely exclusively on useof mobile phone or utensil. As long as the person does not remove thesmart watch, the smart watch goes with them where ever they go andcontinually monitors for possible food consumption activity. Also,continually monitoring wrist motion is far less-intrusive with respectto a person's privacy than continually monitoring what the person sees(video monitoring) or hears (sound monitoring).

In this example, a smart watch 201 collects primary data concerningprobable food consumption and prompts the person to collect secondaryfor food identification when primary data indicates that the person isprobably eating food and the person has not yet collected secondarydata. In this example, primary data is body motion data and secondarydata comprises chemical analysis of food. In this example, smart watch201 is the mechanism for collection of primary data and smart spoon 101is the mechanism for collection of secondary data. In this example,collection of primary data is automatic, not requiring any action by theperson in association with a particular eating event apart from theactual act of eating, but collection of secondary data requires aspecific action (using the smart spoon to carry food) in associationwith a particular eating event apart from the actual act of eating. Inthis example, this combination of automatic primary data collection andnon-automatic secondary data collection combine to provide relativelyhigh-accuracy and high-compliance food consumption measurement withrelatively low privacy intrusion. This is an advantage over foodconsumption devices and methods in the prior art.

In an example, information concerning a person's consumption of at leastone selected type of food, ingredient, and/or nutrient can be combinedwith information from a separate caloric expenditure monitoring devicethat measures a person's caloric expenditure to comprise an overallsystem for energy balance, fitness, weight management, and healthimprovement. In an example, a food-consumption monitoring device (suchas this smart watch) can be in wireless communication with a separatefitness monitoring device. In an example, capability for monitoring foodconsumption can be combined with capability for monitoring caloricexpenditure within a single smart watch device. In an example, a smartwatch device can be used to measure the types and amounts of food,ingredients, and/or nutrients that a person consumes as well as thetypes and durations of the calorie-expending activities in which theperson engages.

13. Narrative to Accompany FIGS. 5 Through 8

The embodiment of this invention that is shown in FIGS. 5 through 8 issimilar to the one that was just shown in FIGS. 1 through 4, except thatnow food is identified by taking pictures of food rather than bychemical analysis of food. In FIGS. 5 through 8, smart spoon 501 of thisdevice and system has a built-in camera 502. In an example, camera 502can be used to take pictures of a mouthful of food 208 in the scoopportion of smart spoon 501. In another example, camera 502 can be usedto take pictures of food before it is apportioned by the spoon, such aswhen food is still on a plate, in a bowl, or in original packaging. Inan example, the types and amounts of food consumed can be identified, ina manner that is at least partially automated, by analysis of foodpictures.

Like the example that was just shown in FIGS. 1 through 4, the examplethat is now shown in FIGS. 5 through 8 shows how this invention can beembodied in a device and system for measuring a person's consumptionthat includes both a wearable food-consumption monitor (a smart watch inthis example) and a hand-held food-identifying sensor (a smart spoon inthis example). However, in this present example, instead of smart spoon101 having a chemical composition sensor 102 that analyzes the chemicalcomposition of food, smart spoon 501 has a camera 502 to takeplain-light pictures of food. These pictures are then analyzed, in amanner that is at least partially automated, in order to identify theamounts and types of foods, ingredients, and/or nutrients that theperson consumes. In an example, these pictures of food can bestill-frame pictures. In an example, these pictures can be motion(video) pictures.

We now discuss the components of the example shown in FIGS. 5 through 8in more detail. In FIG. 5, smart spoon 501 includes camera 502 inaddition to a data processing unit 503, a communication unit 504, and apower supply and/or transducer 50. The latter three components are likethose in the prior example, but the food-identifying sensor (camera 502vs. chemical composition sensor 102) is different. In this example,camera 502 is built into smart spoon 501 and is located on the portionof smart spoon 501 between the spoon's scoop and the portion of thehandle that is held by the person's hand 206.

In this example, camera 502 can be focuses in different directions asthe person moves smart spoon 501. In an example, camera 502 can take apicture of a mouthful of food 208 in the scoop of spoon 501. In anexample, camera 502 can be directed to take a picture of food on aplate, in a bowl, or in packaging. In this example, camera 502 isactivated by touch. In an example, camera 502 can be activated by voicecommand or by motion of smart spoon 501.

FIG. 6 shows smart spoon 501 in use for food consumption, along withsmart watch 201. Smart watch 201 in this example is like smart watch 201shown in the previous example in FIGS. 1 through 4. As in the lastexample, smart watch 201 in FIG. 6 includes communication unit 202,motion sensor 203, data processing unit 204, and power supply and/ortransducer 205. As in the last example, when the person starts movingtheir wrist and arm in the distinctive movements that are associatedwith food consumption, then these movements are recognized by motionsensor 203 on smart watch 201. This is shown in FIG. 7.

If the person has not already used camera 502 on smart spoon 501 to takepictures of food during a particular eating event detected by smartwatch 201, then smart watch 201 prompts the person to take a picture offood using camera 502 on smart spoon 501. In this example, this prompt301 is represented by a “lightning bolt” symbol in FIG. 7. In thisexample, the person complies with prompt 301 and activates camera 502 bytouch in FIG. 8. In this example, a picture is taken of a mouthful offood 208 in the scoop of smart spoon 501. In another example, the personcould aim camera 502 on smart spoon 501 toward food on a plate, food ina bowl, or food packaging to take a picture of food before it isapportioned by spoon 501.

In this example, smart watch 201 collects primary data concerningprobable food consumption and prompts the person to collect secondaryfor food identification when primary data indicates that the person isprobably eating food and the person has not yet collected secondarydata. In this example, primary data is body motion data and secondarydata comprises pictures of food. In this example, smart watch 201 is themechanism for collecting primary data and smart spoon 101 is themechanism for collecting secondary data. In this example, collection ofprimary data is automatic, not requiring any action by the person inassociation with a particular eating event apart from the actual act ofeating, but collection of secondary data requires a specific action(triggering and possibly aiming the camera) in association with aparticular eating event apart from the actual act of eating. In thisexample, automatic primary data collection and non-automatic secondarydata collection combine to provide relatively high-accuracy andhigh-compliance food consumption measurement with relatively low privacyintrusion. This is an advantage over food consumption devices andmethods in the prior art.

In an example, this device and system can prompt a person to use smartspoon 501 for eating and once the person is using smart spoon 501 foreating this spoon can automatically take pictures of mouthfuls of foodthat are in the spoon's scoop. In an example, such automatic picturetaking can be triggered by infrared reflection, other optical sensor,pressure sensor, electromagnetic sensor, or other contact sensor in thespoon scoop. In another example, this device can prompt a person tomanually trigger camera 502 to take a picture of food in the spoon'sscoop. In another example, this device can prompt a person to aim camera502 toward food on a plate, in a bowl, or in original packaging to takepictures of food before it is apportioned into mouthfuls by the spoon.In an example, food on a plate, in a bowl, or in original packaging canbe easier to identify by analysis of its shape, texture, scale, andcolors than food apportioned into mouthfuls.

In an example, use of camera 502 in smart spoon 501 can rely on havingthe person manually aim and trigger the camera for each eating event. Inan example, the taking of food pictures in this manner requires at leastone specific voluntary human action associated with each foodconsumption event, apart from the actual act of eating, in order to takepictures of food during that food consumption event. In an example, suchspecific voluntary human actions can be selected from the groupconsisting of: bringing smart spoon 501 to a meal or snack; using smartspoon 501 to eat food; aiming camera 502 of smart spoon 501 at food on aplate, in a bowl, or in original packaging; triggering camera 502 bytouching a button, screen, or other activation surface; and triggeringcamera 502 by voice command or gesture command.

In an example, camera 502 of smart spoon 501 can be used to takemultiple still-frame pictures of food. In an example, camera 502 ofsmart spoon 501 can be used to take motion (video) pictures of food frommultiple angles. In an example, camera 502 can take pictures of foodfrom at least two different angles in order to better segment a pictureof a multi-food meal into different types of foods, better estimate thethree-dimensional volume of each type of food, and better control fordifferences in lighting and shading. In an example, camera 502 can takepictures of food from multiple perspectives to create a virtualthree-dimensional model of food in order to determine food volume. In anexample, quantities of specific foods can be estimated from pictures ofthose foods by volumetric analysis of food from multiple perspectivesand/or by three-dimensional modeling of food from multiple perspectives.

In an example, pictures of food on a plate, in a bowl, or in packagingcan be taken before and after consumption. In an example, the amount offood that a person actually consumes (not just the amount ordered by theperson or served to the person) can be estimated by measuring thedifference in food volume from pictures before and after consumption. Inan example, camera 502 can image or virtually create a fiduciary marketto better estimate the size or scale of food. In an example, camera 502can be used to take pictures of food which include an object of knownsize. This object can serve as a fiduciary marker in order to estimatethe size and/or scale of food. In an example, camera 502, or anothercomponent on smart spoon 501, can project light beams within the fieldof vision to create a virtual fiduciary marker. In an example, picturescan be taken of multiple sequential mouthfuls of food being transportedby the scoop of smart spoon 501 and used to estimate the cumulativeamount of food consumed.

In an example, there can be a preliminary stage of processing oranalysis of food pictures wherein image elements and/or attributes areadjusted, normalized, or standardized. In an example, a food picture canbe adjusted, normalized, or standardized before it is compared with foodpictures in a food database. This can improve segmentation of a mealinto different types of food, identification of foods, and estimation offood volume or mass. In an example, food size or scale can be adjusted,normalized, or standardized before comparison with pictures in a fooddatabase. In an example, food texture can be adjusted, normalized, orstandardized before comparison with pictures in a food database. In anexample, food lighting or shading can be adjusted, normalized, orstandardized before comparison with pictures in a food database. Invarious examples, a preliminary stage of food picture processing and/oranalysis can include adjustment, normalization, or standardization offood color, texture, shape, size, context, geographic location, adjacentfoods, place setting context, and temperature.

In an example, a food database can be used as part of a device andsystem for identifying types and amounts of food, ingredients, ornutrients. In an example, a food database can include one or moreelements selected from the group consisting of: food name, food picture(individually or in combinations with other foods), food color, foodpackaging bar code or nutritional label, food packaging or logo pattern,food shape, food texture, food type, common geographic or intra-buildinglocations for serving or consumption, common or standardized ingredients(per serving, per volume, or per weight), common or standardizednutrients (per serving, per volume, or per weight), common orstandardized size (per serving), common or standardized number ofcalories (per serving, per volume, or per weight), common times orspecial events for serving or consumption, and commonly associated orjointly-served foods.

In an example, the boundaries between different types of food in apicture of a meal can be automatically determined to segment the mealinto different food types before comparison with pictures in a fooddatabase. In an example, individual portions of different types of foodwithin a multi-food meal can be compared individually with images ofportions of different types of food in a food database. In an example, apicture of a meal including multiple types of food can be automaticallysegmented into portions of different types of food for comparison withdifferent types of food in a food database. In an example, a picture ofa meal with multiple types of food can be compared as a whole withpictures of meals with multiple types of food in a food database.

In an example, a food database can also include average amounts ofspecific ingredients and/or nutrients associated with specific types andamounts of foods for measurement of at least one selected type ofingredient or nutrient. In an example, a food database can be used toidentify the type and amount of at least one selected type of ingredientthat is associated with an identified type and amount of food. In anexample, a food database can be used to identify the type and amount ofat least one selected type of nutrient that is associated with anidentified type and amount of food. In an example, an ingredient ornutrient can be associated with a type of food on a per-portion,per-volume, or per-weight basis.

In an example, automatic identification of food amounts and types caninclude extracting a vector of food parameters (such as color, texture,shape, and size) from a food picture and comparing this vector withvectors of these parameters in a food database. In various examples,methods for automatic identification of food types and amounts from foodpictures can include: color analysis, image pattern recognition, imagesegmentation, texture analysis, three-dimensional modeling based onpictures from multiple perspectives, and volumetric analysis based on afiduciary marker or other object of known size.

In various examples, food pictures can be analyzed in a manner which isat least partially automated in order to identify food types and amountsusing one or more methods selected from the group consisting of:analysis of variance; chi-squared analysis; cluster analysis; comparisonof a vector of food parameters with a food database containing suchparameters; energy balance tracking; factor analysis; Fouriertransformation and/or fast Fourier transform (FFT); image attributeadjustment or normalization; pattern recognition; comparison with foodimages with food images in a food database; inter-food boundarydetermination and food portion segmentation; linear discriminantanalysis; linear regression and/or multivariate linear regression;logistic regression and/or probit analysis; neural network and machinelearning; non-linear programming; principal components analysis; scaledetermination using a physical or virtual fiduciary marker;three-dimensional modeling to estimate food quantity; time seriesanalysis; and volumetric modeling.

In an example, attributes of food in an image can be represented by amulti-dimensional food attribute vector. In an example, this foodattribute vector can be statistically compared to the attribute vectorof known foods in order to automate food identification. In an example,multivariate analysis can be done to identify the most likelyidentification category for a particular portion of food in an image. Invarious examples, a multi-dimensional food attribute vector can includeattributes selected from the group consisting of: food color; foodtexture; food shape; food size or scale; geographic location ofselection, purchase, or consumption; timing of day, week, or specialevent; common food combinations or pairings; image brightness,resolution, or lighting direction; infrared light reflection;spectroscopic analysis; and person-specific historical eating patterns.In an example, in some situations the types and amounts of food can beidentified by analysis of bar codes, brand logos, nutritional labels, orother optical patterns on food packaging.

In an example, analysis of data concerning food consumption can includecomparison of food consumption parameters between a specific person anda reference population. In an example, data analysis can includeanalysis of a person's food consumption patterns over time. In anexample, such analysis can track the cumulative amount of at least oneselected type of food, ingredient, or nutrient that a person consumesduring a selected period of time.

In an example, pictures of food can be analyzed within the dataprocessing unit of a hand-held device (such as a smart spoon) or awearable device (such as a smart watch). In an example, pictures of foodcan be wirelessly transmitted from a hand-held or wearable device to anexternal device, wherein these food pictures are automatically analyzedand food identification occurs. In an example, the results of foodidentification can then be wirelessly transmitted back to the wearableor hand-held device. In an example, identification of the types andquantities of foods, ingredients, or nutrients that a person consumescan be a combination of, or interaction between, automatedidentification food methods and human-based food identification methods.

In the example shown in FIGS. 5 through 8, food-imaging camera 502 isbuilt into smart spoon 501. In various alternative examples, a deviceand system for measuring a person's consumption of at least one selectedtype of food, ingredient, or nutrient can take pictures of food with animaging device or component that is selected from the group consistingof: smart food utensil and/or electronically-functional utensil, smartspoon, smart fork, food probe, smart chop stick, smart plate, smartdish, or smart glass; smart phone, mobile phone, or cell phone; smartwatch, watch cam, smart bracelet, fitness watch, fitness bracelet, watchphone, or bracelet phone; smart necklace, necklace cam, smart beads,smart button, neck chain, or neck pendant; smart finger ring or ringcam; electronically-functional or smart eyewear, smart glasses, visor,augmented or virtual reality glasses, or electronically-functionalcontact lens; digital camera; and electronic tablet.

14. Narrative to Accompany FIGS. 9 Through 12

The embodiment of this invention that is shown in FIGS. 9 through 12 issimilar to the one that was just shown in FIGS. 5 through 8, except thatnow food pictures are taken by a general-purpose mobile electronicdevice (such as a smart phone) rather than by a specialized food utensil(such as a smart spoon). In this example, the general-purpose mobileelectronic device is a smart phone. In other examples, a general-purposemobile electronic device can be an electronic tablet or a digitalcamera.

The wearable food-monitoring component of the example shown in FIGS. 9through 12 is again a smart watch with a motion sensor, like the one inprevious examples. The smart watch and smart phone components of thisexample work together in FIGS. 9 through 12 in a similar manner to theway in which the smart watch and smart spoon components worked togetherin the example shown in FIGS. 5 through 8. We do not repeat themethodological detail of possible ways to identify food based on foodpictures here because this was already discussed in the narrativeaccompanying the previous example.

FIG. 9 shows a rectangular general-purpose smart phone 901 that includesa camera (or other imaging component) 902. FIG. 10 shows a persongrasping food item 1001 in their hand 206. FIG. 10 also shows that thisperson is wearing a smart watch 201 that includes communication unit202, motion sensor 203, data processing unit 204, and power supplyand/or transducer 205. In an example, food item 1001 can be a deep-friedpork rind. In another example, food item 1001 can be a blob of plaintofu; however, it is unlikely that any person who eats a blob of plaintofu would even need a device like this.

FIG. 11 shows this person bringing food item 1001 up to their mouth witha distinctive rotation of their wrist that is represented by thedotted-line arrow around hand 206. This indicates that the person isprobably eating food. Using motion sensor 203, smart watch 201 detectsthis pattern of movement and detects that the person is probably eatingsomething. Since the person has not yet taken a picture of food inassociation with this eating event, smart watch 201 prompts the personto take a picture of food using smart phone 901. This prompt 301 isrepresented in FIG. 11 by a “lightning bolt” symbol coming out fromcommunication unit 202. We discussed a variety of possible prompts inearlier examples and do not repeat them here.

FIG. 12 shows that this person responds positively to prompt 301. Thisperson responds by taking a picture of food items 1001 in bowl 1201using camera 902 in smart phone 901. The field of vision of camera 902is represented by dotted-line rays 1202 that radiate from camera 902toward bowl 1201. In an example, the person manually aims camera 902 ofsmart phone 901 toward the food source (bowl 1201 in this example) andthen triggers camera 902 to take a picture by touching the screen ofsmart phone 901. In another example, the person could trigger camera 902with a voice command or a gesture command.

In this example, smart watch 201 and smart phone 901 share wirelesscommunication. In an example, communication with smart watch 201 can bepart of a smart phone application that runs on smart phone 901. In anexample, smart watch 201 and smart phone 901 can comprise part of anintegrated system for monitoring and modifying caloric intake andcaloric expenditure to achieve energy balance, weight management, andimproved health.

In an example, smart watch 201 and/or smart phone 901 can also be incommunication with an external computer. An external computer canprovide advanced data analysis, data storage and memory, communicationwith health care professionals, and/or communication with a supportnetwork of friends. In an example, a general purpose smart phone cancomprise the computer-to-human interface of a device and system formeasuring a person's consumption of at least one selected type of food,ingredient, or nutrient. In an example, such a device and system cancommunicate with a person by making calls or sending text messagesthrough a smart phone. In an alternative example, an electronic tabletcan serve the role of a hand-held imaging and interface device insteadof smart phone 901.

FIGS. 9 through 12 show an embodiment of a device for measuring aperson's consumption of at least one selected type of food, ingredient,or nutrient comprising a wearable food-consumption monitor (a smartwatch in this example) that is configured to be worn on the person'swrist, arm, hand or finger and a hand-held food-identifying sensor (asmart phone in this example). The person is prompted to use the smartphone to take pictures of food when the smart watch indicates that theperson is consuming food. In this example, primary data concerning foodconsumption that is collected by a smart watch includes data concerningmovement of the person's body and secondary data for food identificationthat is collected by a smart phone includes pictures of food. In thisexample, the person is prompted to take pictures of food when they aremoving in a manner that indicates that they are probably eating andsecondary data has not already been collected.

The system for measuring food consumption that is shown in FIGS. 9through 12 combines continual motion monitoring by a smart watch andfood imaging by a smart phone. It is superior to prior art that reliesonly on a smart phone. A system for measuring food consumption thatdepends only on the person using a smart phone to take a picture ofevery meal and every snack they eat will probably have much lowercompliance and accuracy than the system disclosed herein. With thesystem disclosed herein, as long as the person wears the smart watch(which can be encouraged by making it comfortable and tamper resistant),the system disclosed herein continually monitors for food consumption. Asystem based on a stand-alone smart phone offers no such functionality.

Ideally, if the smart watch 201 herein is designed to be sufficientlycomfortable and unobtrusive, it can be worn all the time. Accordingly,it can even monitor for night-time snacking. It can monitor foodconsumption at times when a person would be unlikely to bring out theirsmart phone to take pictures (at least not without prompting). Thefood-imaging device and system that is shown here in FIGS. 9 through 12,including the coordinated operation of a motion-sensing smart watch anda wirelessly-linked smart phone, can provide highly-accurate foodconsumption measurement with relatively-low privacy intrusion.

FIGS. 9 through 12 also show an example of how this invention can beembodied in a device for monitoring food consumption comprising: (a) awearable sensor that is configured to be worn on a person's body orclothing, wherein this wearable sensor automatically collects data thatis used to detect probable eating events without requiring action by theperson in association with a probable eating event apart from the act ofeating, and wherein a probable eating event is a period of time duringwhich the person is probably eating; (b) an imaging member, wherein thisimaging member is used by the person to take pictures of food that theperson eats, wherein using this imaging member to take pictures of foodrequires voluntary action by the person apart from the act of eating,and wherein the person is prompted to take pictures of food using thisimaging member when data collected by the wearable sensor indicates aprobable eating event; and (c) a data analysis component, wherein thiscomponent analyzes pictures of food taken by the imaging member toestimate the types and amounts of foods, ingredients, nutrients, and/orcalories that are consumed by the person. In this example, the wearablesensor is motion sensor 203. In this example, the imaging member iscamera 902 which is part of phone 901. In this example, the dataanalysis component is data processing unit 204.

In the example shown in FIGS. 9 through 12, motion sensor 203automatically collects data that is used to detect probable eatingevents. In this example, this data comprises hand motion. When datacollected by motion sensor 203 indicates a probable eating event, thencommunication unit 202 sends a signal that prompts the person to useimaging member 902 to take pictures of food 1001 which the person iseating. When prompted, the person uses camera 902 to take pictures offood 1001. Then, data analysis component 204 analyzes these foodpictures to estimate the types and amounts of foods, ingredients,nutrients, and/or calories that are consumed by the person.

In this example, data analysis occurs in a wrist-based data analysiscomponent. In other examples, analysis of food pictures can occur inother locations. In an example, analysis of food pictures can occur in adata analysis component that is located in phone 901. In anotherexample, analysis of food pictures can occur in a remote computer withwhich phone 901 or communication unit 202 is in wireless communication.

In the example shown in FIGS. 9 through 12, a wearable sensor is worn onthe person's wrist. In other examples, a wearable sensor can be worn ona person's hand, finger, or arm. In this example, a wearable sensor ispart of an electronically-functional wrist band or smart watch. Inanother example, a wearable sensor can be an electronically-functionaladhesive patch that is worn on a person's skin. In another example, asensor can be worn on a person's clothing.

In the example shown in FIGS. 9 through 12, an imaging member is amobile phone or mobile phone application. In another example, an imagingmember can be electronically-functional eyewear. In another example, animaging member can be a smart watch. In another example, an imagingmember can be an electronically-functional necklace. In this example, awearable sensor and imaging member are separate but in wirelesscommunication with each other. In another example, a wearable sensor andan imaging member can be jointly located, such as in a smart watch,necklace, or eyewear.

In the example shown in FIGS. 9 through 12, a wearable sensorautomatically collects data concerning motion of the person's body. Inanother example, a wearable sensor can automatically collect dataconcerning electromagnetic energy that is emitted from the person's bodyor transmitted through the person's body. In another example, a wearablesensor can automatically collect data concerning thermal energy that isemitted from the person's body. In another example, a wearable sensorcan automatically collect data concerning light energy that is reflectedfrom the person's body or absorbed by the person's body. In variousexamples, food events can be detected by monitoring selected from thegroup consisting of: monitoring motion of the person's body; monitoringelectromagnetic energy that is emitted from the person's body ortransmitted through the person's body; monitoring thermal energy that isemitted from the person's body; and monitoring light energy that isreflected from the person's body or absorbed by the person's body.

In the example shown in FIGS. 9 through 12, the person is prompted totake pictures of food using the imaging member when data collected bythe wearable sensor indicates a probable eating event and the persondoes not take pictures of food for this probable eating event before orat the start of the probable eating event. In an example, the person canbe prompted to take pictures of food using the imaging member when datacollected by the wearable sensor indicates a probable eating event andthe person does not take pictures of food for this probable eating eventbefore a selected length of time after the start of the probable eatingevent. In an example, the person can be prompted to take pictures offood using the imaging member when data collected by the wearable sensorindicates a probable eating event and the person does not take picturesof food for this probable eating event before a selected quantity ofeating-related actions occurs during the probable eating event. In anexample, the person can be prompted to take pictures of food using theimaging member when data collected by the wearable sensor indicates aprobable eating event and the person does not take pictures of food forthis probable eating event at the end of the probable eating event.

In a variation on this example, this invention can be embodied in adevice for monitoring food consumption comprising: (a) a wearable sensorthat is configured to be worn on a person's wrist, hand, finger, or arm,wherein this wearable sensor automatically collects data that is used todetect probable eating events without requiring action by the person inassociation with a probable eating event apart from the act of eating,and wherein a probable eating event is a period of time during which theperson is probably eating; (b) an imaging member, wherein this imagingmember is used by the person to take pictures of food that the personeats, wherein using this imaging member to take pictures of foodrequires voluntary action by the person apart from the act of eating,and wherein the person is prompted to take pictures of food using thisimaging member when data collected by the wearable sensor indicates aprobable eating event; and (c) a data analysis component, wherein thiscomponent analyzes pictures of food taken by the imaging member toestimate the types and amounts of foods, ingredients, nutrients, and/orcalories that are consumed by the person.

In a variation on this example, this invention can be embodied in adevice for monitoring food consumption comprising: (a) a wearable sensorthat is configured to be worn on a person's wrist, hand, finger, or arm,wherein this wearable sensor automatically collects data that is used todetect probable eating events without requiring action by the person inassociation with a probable eating event apart from the act of eating,wherein a probable eating event is a period of time during which theperson is probably eating, and wherein this data is selected from thegroup consisting of data concerning motion of the person's body, dataconcerning electromagnetic energy emitted from or transmitted throughthe person's body, data concerning thermal energy emitted from theperson's body, and light energy reflected from or absorbed by theperson's body; (b) an imaging member, wherein this imaging member isused by the person to take pictures of food that the person eats,wherein using this imaging member to take pictures of food requiresvoluntary action by the person apart from the act of eating, wherein theperson is prompted to take pictures of food using this imaging memberwhen data collected by the wearable sensor indicates a probable eatingevent; and (c) a data analysis component, wherein this componentanalyzes pictures of food taken by the imaging member to estimate thetypes and amounts of foods, ingredients, nutrients, and/or calories thatare consumed by the person, and wherein this component analyzes datareceived from the sensor and pictures of food taken by the imagingmember to evaluate the completeness of pictures taken by the imagingmember for tracking the person's total food consumption.

15. Narrative to Accompany FIGS. 13 Through 18

The embodiment of this invention that is shown in FIGS. 13 through 15 issimilar to the one that was just shown in FIGS. 9 through 12, exceptthat the wearable food-monitoring component is now a smart necklaceinstead of a smart watch. The smart necklace in this example monitorsfor food consumption by monitoring sounds instead of motion. In thisexample, the smart necklace detects food consumption by detectingchewing or swallowing sounds.

FIG. 13 shows the smart phone 901 with camera 902 that was introduced inthe previous example. FIG. 14 shows that the person 1401 is wearingsmart necklace 1402 including communication unit 1403, data processingunit and power supply 1404, and microphone 1405. FIG. 14 also shows thatthe person is eating food item 1001 using fork 1406.

In FIG. 14, microphone 1405 of smart necklace 1402 detects that theperson is consuming food based on chewing or swallowing sounds. In FIG.14, chewing or swallowing sounds are represented by dotted-line curves1407 expanding outwardly from the person's mouth. Smart necklace 1402then prompts the person to take a picture of food using camera 902 onsmart phone 901. In FIG. 14, this prompt 1408 is represented by a“lightning bolt” symbol coming out from communication unit 1403.

FIG. 15 shows that the person responds to prompt 1408 by aiming camera902 of smart phone 901 toward bowl 1201 containing food items 1001. Thefield of vision of camera 902 is represented by dotted-line rays 1202that radiate outwards from camera 902 toward bowl 1201.

The embodiment of this invention that is shown in FIGS. 16 through 18 issimilar to the one that was just shown in FIGS. 13 through 15, exceptthat hand-held food-identifying component is the smart spoon that wasintroduced earlier instead of a smart phone. FIG. 16 shows smart spoon101 with chemical composition sensor 102, data processing unit 103,communication unit 104, and power supply and/or transducer 105.

FIG. 17 shows that the person is eating food item 1001 without usingsmart spoon 101. In FIG. 17, microphone 1405 of smart necklace 1402detects that the person is consuming food based on chewing or swallowingsounds 1407. In FIG. 14, chewing or swallowing sounds are represented bydotted-line curves 1407 expanding outwardly from the person's mouth.Smart necklace 1402 then prompts the person to use smart spoon 101 toeat food item 1001. In FIG. 14, this prompt 1408 is represented by a“lightning bolt” symbol coming out from communication unit 1403.

FIG. 18 shows that the person responds to prompt 1408 by using smartspoon 101. Use of smart spoon 101 brings food item 1001 into contactwith chemical composition sensor 102 on smart spoon 101. This contactenables identification of food item 1001.

16. Conclusion of Narrative for Figures

FIGS. 1 through 18 show various examples of a device for measuring aperson's consumption of at least one selected type of food, ingredient,or nutrient comprising: a wearable food-consumption monitor, whereinthis food-consumption monitor is configured to be worn on a person'sbody or clothing, and wherein this food-consumption monitorautomatically collects primary data that is used to detect when a personis consuming food, without requiring any specific action by the personin association with a specific eating event with the exception of theact of eating; and a hand-held food-identifying sensor, wherein thisfood-identifying sensor collects secondary data that is used to identifythe person's consumption of at least one selected type of food,ingredient, or nutrient.

In FIGS. 1 through 18, the collection of secondary data by a hand-heldfood-identifying sensor requires a specific action by the person inassociation with a specific eating event apart from the act of eating.Also in FIGS. 1 through 18, the person whose food consumption ismonitored is prompted to perform a specific action to collect secondarydata when primary data collected by a food-consumption monitor indicatesthat the person is probably eating and the person has not alreadycollected secondary data in association with a specific eating event.

FIGS. 1 through 12 show various examples of a device wherein a wearablefood-consumption monitor is a smart watch or smart bracelet. FIGS. 9through 15 show various examples of a device wherein a hand-heldfood-identifying sensor is a smart phone, cell phone, or mobile phone.FIGS. 1 through 8 and also FIGS. 16 through 18 show various examples ofa device wherein a hand-held food-identifying sensor is a smart fork,smart spoon, other smart utensil, or food probe.

FIGS. 1 through 4 show an example of a device wherein a wearablefood-consumption monitor is a smart watch or other electronic memberthat is configured to be worn on the person's wrist, arm, hand orfinger; wherein a hand-held food-identifying sensor is a smart foodutensil or food probe; and wherein a person is prompted to use the smartfood utensil or food probe to analyze the chemical composition of foodwhen the smart watch indicates that the person is consuming food.

FIGS. 1 through 4 show an example of a device wherein a wearablefood-consumption monitor is a smart watch or other electronic memberthat is configured to be worn on the person's wrist, arm, hand orfinger; wherein primary data collected by the smart watch or otherelectronic member that is configured to be worn on the person's wrist,arm, hand or finger includes data concerning movement of the person'sbody; wherein a hand-held food-identifying sensor is a smart foodutensil or food probe; and wherein a person is prompted to use the smartfood utensil or food probe to analyze the chemical composition of foodwhen the smart watch indicates that the person is consuming food.

FIGS. 9 through 12 show an example of a device wherein a wearablefood-consumption monitor is a smart watch or other electronic memberthat is configured to be worn on the person's wrist, arm, hand orfinger; wherein a hand-held food-identifying sensor is a smart phone,cell phone, or mobile phone; and wherein a person is prompted to use thesmart phone, cell phone, or mobile phone to take pictures of food orfood packaging when the smart watch indicates that the person isconsuming food.

FIGS. 9 through 12 show an example of a device wherein a wearablefood-consumption monitor is a smart watch or other electronic memberthat is configured to be worn on the person's wrist, arm, hand orfinger; wherein primary data collected by the smart watch or otherelectronic member that is configured to be worn on the person's wrist,arm, hand or finger includes data concerning movement of the person'sbody; wherein a hand-held food-identifying sensor is a smart phone, cellphone, or mobile phone; and wherein a person is prompted to use thesmart phone, cell phone, or mobile phone to take pictures of food orfood packaging when primary data indicates that the person is consumingfood.

In another example: a wearable food-consumption monitor can be a smartwatch or other electronic member that is configured to be worn on theperson's wrist, arm, hand or finger wherein primary data collected bythe smart watch or other electronic member that is configured to be wornon the person's wrist, arm, hand or finger includes data concerningelectromagnetic energy received from the person's body; a hand-heldfood-identifying sensor can be a smart food utensil or food probe; and aperson can be prompted to use the smart food utensil or food probe toanalyze the chemical composition of food when the smart watch indicatesthat the person is consuming food.

In another example: a wearable food-consumption monitor can be a smartwatch or other electronic member that is configured to be worn on theperson's wrist, arm, hand or finger wherein primary data collected bythe smart watch or other electronic member that is configured to be wornon the person's wrist, arm, hand or finger includes data concerningelectromagnetic energy received from the person's body; a hand-heldfood-identifying sensor can be a smart phone, cell phone, or mobilephone; and a person can be prompted to use the smart phone, cell phone,or mobile phone to take pictures of food or food packaging when primarydata indicates that the person is consuming food.

In another example: a wearable food-consumption monitor can be a smartwatch or other electronic member that is configured to be worn on theperson's wrist, arm, hand or finger wherein primary data collected bythe smart watch or other electronic member that is configured to be wornon the person's wrist, arm, hand or finger includes images; a hand-heldfood-identifying sensor can be a smart food utensil or food probe; and aperson can be prompted to use the smart food utensil or food probe toanalyze the chemical composition of food when the smart watch indicatesthat the person is consuming food.

In another example: a wearable food-consumption monitor can be a smartwatch or other electronic member that is configured to be worn on theperson's wrist, arm, hand or finger wherein primary data collected bythe smart watch or other electronic member that is configured to be wornon the person's wrist, arm, hand or finger includes images; a hand-heldfood-identifying sensor can be a smart phone, cell phone, or mobilephone; and a person can be prompted to use the smart phone, cell phone,or mobile phone to take pictures of food or food packaging when primarydata indicates that the person is consuming food.

In another example: a wearable food-consumption monitor is a smartnecklace or other electronic member that is configured to be worn on theperson's neck, head, or torso wherein primary data collected by thesmart watch or other electronic member that is configured to be worn onthe person's wrist, arm, hand or finger includes patterns of sonicenergy; a hand-held food-identifying sensor can be a smart food utensilor food probe; and a person can be prompted to use the smart foodutensil or food probe to analyze the chemical composition of food whenthe smart watch indicates that the person is consuming food.

In another example: a wearable food-consumption monitor is a smartnecklace or other electronic member that is configured to be worn on theperson's neck, head, or torso wherein primary data collected by thesmart watch or other electronic member that is configured to be worn onthe person's wrist, arm, hand or finger includes patterns of sonicenergy; a hand-held food-identifying sensor can be a smart phone, cellphone, or mobile phone; and a person can be prompted to use the smartphone, cell phone, or mobile phone to take pictures of food or foodpackaging when primary data indicates that the person is consuming food.

In an example, at least one selected type of food, ingredient, ornutrient for these examples can be selected from the group consistingof: a specific type of carbohydrate, a class of carbohydrates, or allcarbohydrates; a specific type of sugar, a class of sugars, or allsugars; a specific type of fat, a class of fats, or all fats; a specifictype of cholesterol, a class of cholesterols, or all cholesterols; aspecific type of protein, a class of proteins, or all proteins; aspecific type of fiber, a class of fiber, or all fiber; a specificsodium compound, a class of sodium compounds, and all sodium compounds;high-carbohydrate food, high-sugar food, high-fat food, fried food,high-cholesterol food, high-protein food, high-fiber food, andhigh-sodium food.

In an example, at least one selected type of food, ingredient, ornutrient can be selected from the group consisting of: a selected food,ingredient, or nutrient that has been designated as unhealthy by ahealth care professional organization or by a specific health careprovider for a specific person; a selected substance that has beenidentified as an allergen for a specific person; peanuts, shellfish, ordairy products; a selected substance that has been identified as beingaddictive for a specific person; alcohol; a vitamin or mineral; vitaminA, vitamin B1, thiamin, vitamin B12, cyanocobalamin, vitamin B2,riboflavin, vitamin C, ascorbic acid, vitamin D, vitamin E, calcium,copper, iodine, iron, magnesium, manganese, niacin, pantothenic acid,phosphorus, potassium, riboflavin, thiamin, and zinc; a specific type ofcarbohydrate, class of carbohydrates, or all carbohydrates; a specifictype of sugar, class of sugars, or all sugars; simple carbohydrates,complex carbohydrates; simple sugars, complex sugars, monosaccharides,glucose, fructose, oligosaccharides, polysaccharides, starch, glycogen,disaccharides, sucrose, lactose, starch, sugar, dextrose, disaccharide,fructose, galactose, glucose, lactose, maltose, monosaccharide,processed sugars, raw sugars, and sucrose; a specific type of fat, classof fats, or all fats; fatty acids, monounsaturated fat, polyunsaturatedfat, saturated fat, trans fat, and unsaturated fat; a specific type ofcholesterol, a class of cholesterols, or all cholesterols; Low DensityLipoprotein (LDL), High Density Lipoprotein (HDL), Very Low DensityLipoprotein (VLDL), and triglycerides; a specific type of protein, aclass of proteins, or all proteins; dairy protein, egg protein, fishprotein, fruit protein, grain protein, legume protein, lipoprotein, meatprotein, nut protein, poultry protein, tofu protein, vegetable protein,complete protein, incomplete protein, or other amino acids; a specifictype of fiber, a class of fiber, or all fiber; dietary fiber, insolublefiber, soluble fiber, and cellulose; a specific sodium compound, a classof sodium compounds, and all sodium compounds; salt; a specific type ofmeat, a class of meats, and all meats; a specific type of vegetable, aclass of vegetables, and all vegetables; a specific type of fruit, aclass of fruits, and all fruits; a specific type of grain, a class ofgrains, and all grains; high-carbohydrate food, high-sugar food,high-fat food, fried food, high-cholesterol food, high-protein food,high-fiber food, and high-sodium food.

FIGS. 1 through 18 show various examples of a device for measuring aperson's consumption of at least one selected type of food, ingredient,or nutrient comprising: (a) a wearable food-consumption monitor, whereinthis food-consumption monitor is configured to be worn on a person'sbody or clothing, and wherein this food-consumption monitorautomatically collects primary data that is used to detect when a personis consuming food, without requiring any specific action by the personin association with a specific eating event with the exception of theact of eating; (b) a hand-held food-identifying sensor, wherein thisfood-identifying sensor collects secondary data that is used to identifythe person's consumption of at least one selected type of food,ingredient, or nutrient; wherein collection of secondary data by thishand-held food-identifying sensor requires a specific action by theperson in association with a specific eating event apart from the act ofeating; and (c) a computer-to-human interface, wherein this interfaceuses visual, auditory, tactile, electromagnetic, gustatory, and/orolfactory communication to prompt the person to use the hand-heldfood-identifying sensor to collect secondary data when primary datacollected by the food-consumption monitor indicates that the person isprobably eating and the person has not already collected secondary datain association with a specific eating event.

FIGS. 1 through 18 also show various examples of a method for measuringa person's consumption of at least one selected type of food,ingredient, or nutrient comprising: (a) automatically collecting primarydata using a food-consumption monitor that a person wears on their bodyor clothing without requiring any specific action by the person inassociation with a specific eating event with the possible exception ofthe act of eating, wherein this primary data is used to detect when theperson is consuming food; (b) collecting secondary data using ahand-held food-identifying sensor wherein collection of secondary datarequires a specific action by the person in association with a specificeating event apart from the act of eating, and wherein this secondarydata is used to identify the person's consumption of at least oneselected type of food, ingredient, or nutrient; and (c) prompting theperson to use a hand-held food-identifying sensor to collect secondarydata when primary data collected by a food-consumption monitor indicatesthat the person is eating and the person has not already collectedsecondary data in association with a specific eating event.

Figures shown and discussed herein also disclose a device for monitoringfood consumption comprising: (a) a wearable sensor that is configured tobe worn on a person's body or clothing, wherein this wearable sensorautomatically collects data that is used to detect probable eatingevents without requiring action by the person in association with aprobable eating event apart from the act of eating, and wherein aprobable eating event is a period of time during which the person isprobably eating; (b) an imaging member, wherein this imaging member isused by the person to take pictures of food that the person eats,wherein using this imaging member to take pictures of food requiresvoluntary action by the person apart from the act of eating, and whereinthe person is prompted to take pictures of food using this imagingmember when data collected by the wearable sensor indicates a probableeating event; and (c) a data analysis component, wherein this componentanalyzes pictures of food taken by the imaging member to estimate thetypes and amounts of foods, ingredients, nutrients, and/or calories thatare consumed by the person.

Figures shown and discussed herein disclose a device for monitoring foodconsumption wherein the wearable sensor is worn on a person's wrist,hand, finger, or arm. Figures shown and discussed herein disclose adevice wherein the wearable sensor is part of anelectronically-functional wrist band or smart watch. In another example,a wearable sensor can be part of an electronically-functional adhesivepatch that is worn on a person's skin.

Figures shown and discussed herein disclose a device for monitoring foodconsumption wherein the imaging member is a mobile phone or mobile phoneapplication. In another example, the imaging member can beelectronically-functional eyewear. In another example, the imagingmember can be a smart watch. In another example, the imaging member canbe an electronically-functional necklace. In another example, theimaging member can be an electronically-functional wearable button.

Figures shown and discussed herein disclose a device for monitoring foodconsumption wherein the wearable sensor and the imaging member are inwireless communication with each other. Figures shown and discussedherein disclose a device for monitoring food consumption wherein thewearable sensor automatically collects data concerning motion of theperson's body. In another example, the wearable sensor can automaticallycollect data concerning electromagnetic energy emitted from the person'sbody or transmitted through the person's body. In another example, thewearable sensor can automatically collect data concerning thermal energyemitted from the person's body. In another example, the wearable sensorcan automatically collect data concerning light energy reflected fromthe person's body or absorbed by the person's body.

Figures shown and discussed herein disclose a device for monitoring foodconsumption wherein the person is prompted to take pictures of foodusing the imaging member when data collected by the wearable sensorindicates a probable eating event and the person does not take picturesof food for this probable eating event before or at the start of theprobable eating event. In another example, the person can be prompted totake pictures of food using the imaging member when data collected bythe wearable sensor indicates a probable eating event and the persondoes not take pictures of food for this probable eating event before aselected length of time after the start of the probable eating event. Inanother example, the person can be prompted to take pictures of foodusing the imaging member when data collected by the wearable sensorindicates a probable eating event and the person does not take picturesof food for this probable eating event before a selected quantity ofeating-related actions occurs during the probable eating event. Inanother example, the person can be prompted to take pictures of foodusing the imaging member when data collected by the wearable sensorindicates a probable eating event and the person does not take picturesof food for this probable eating event at the end of the probable eatingevent.

Figures shown and discussed herein also disclose a device for monitoringfood consumption comprising: (a) a wearable sensor that is configured tobe worn on a person's wrist, hand, finger, or arm, wherein this wearablesensor automatically collects data that is used to detect probableeating events without requiring action by the person in association witha probable eating event apart from the act of eating, and wherein aprobable eating event is a period of time during which the person isprobably eating; (b) an imaging member, wherein this imaging member isused by the person to take pictures of food that the person eats,wherein using this imaging member to take pictures of food requiresvoluntary action by the person apart from the act of eating, and whereinthe person is prompted to take pictures of food using this imagingmember when data collected by the wearable sensor indicates a probableeating event; and (c) a data analysis component, wherein this componentanalyzes pictures of food taken by the imaging member to estimate thetypes and amounts of foods, ingredients, nutrients, and/or calories thatare consumed by the person.

Figures shown and discussed herein also disclose a device for monitoringfood consumption comprising: (a) a wearable sensor that is configured tobe worn on a person's wrist, hand, finger, or arm, wherein this wearablesensor automatically collects data that is used to detect probableeating events without requiring action by the person in association witha probable eating event apart from the act of eating, wherein a probableeating event is a period of time during which the person is probablyeating, and wherein this data is selected from the group consisting ofdata concerning motion of the person's body, data concerningelectromagnetic energy emitted from or transmitted through the person'sbody, data concerning thermal energy emitted from the person's body, andlight energy reflected from or absorbed by the person's body; (b) animaging member, wherein this imaging member is used by the person totake pictures of food that the person eats, wherein using this imagingmember to take pictures of food requires voluntary action by the personapart from the act of eating, wherein the person is prompted to takepictures of food using this imaging member when data collected by thewearable sensor indicates a probable eating event; and (c) a dataanalysis component, wherein this component analyzes pictures of foodtaken by the imaging member to estimate the types and amounts of foods,ingredients, nutrients, and/or calories that are consumed by the person,and wherein this component analyzes data received from the sensor andpictures of food taken by the imaging member to evaluate thecompleteness of pictures taken by the imaging member for tracking theperson's total food consumption.

I claim:
 1. A multi-sensor caloric intake monitoring and modifyingsystem comprising: a smart watch or wrist band, wherein the smart watchor wrist band is worn by a person; a necklace or neck band, wherein thenecklace or neck band is worn by the person; a spectroscopic sensor thatis part of the smart watch or wrist band, wherein the spectroscopicsensor collects data concerning light that is absorbed by or reflectedfrom food; a camera that is part of the necklace or neck band, whereinthe camera takes pictures of the food; a microphone that is part of thenecklace or neck band, wherein the microphone monitors sounds associatedwith eating the food; a data processor which analyzes data concerninglight that is absorbed by or reflected from the food from thespectroscopic sensor, pictures of the food from the camera, and soundsassociated with eating the food from the microphone, wherein analyzingthe data comprises: creating multi-dimensional food attribute vectors,identifying the food based on the vector, and monitoring the person'sfood consumption based on the identified food; and a buzzer, vibrator,or alarm that is part of the smart watch, wrist band, necklace, or neckband, wherein the buzzer, vibrator, or alarm provides auditory and/ortactile feedback to the person in order to modify the person's foodconsumption.